Automated database management

Quality input for quality output. Create your own essential datasets with custom data aggregation algorithms.

Generate databases from APIs

Many organisations (from the FAO, to the OECD, to Twitter or the World Bank, for example) make vast amounts of data available through application programming interfaces (“APIs”) that allow for the compilation of datasets for any kind of purpose. Once you have identified potentially valuable datasets, the generation of databases to include in your own applications can be automated to make sure you always have the latest available data at your disposal.

Databases from web scraping

Often times, data is freely available on websites, but in human readable formats rather than machine readable formats. Such data can nevertheless be compiled into useful databases if their pulication is systematic enough to warrant “scraping”. For example, government organs may publish regular updates about work programmes or contact information, which could be automatically “scraped” and fed into an always up-to-date database.

Making optimal use of databases

Once you have generated databases, these can be integrated for use in different ways, whether as Excel sheets for people to use as they see fit, or in an appropriate format for other programmes or apps to redeploy according to their defined needs (see also automated monitoring, automated data analysis, automated data visualisation, automated publishing, automated reporting).

Start automating today
This site makes use of essential cookies. By continuing to navigate this site, you agree to our Cookie Policy and Terms of Service. This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.Accept & continue