data warehouse
The data storage system that analyses and reports on data such as a data mart.
digital
Data engineers develop the architecture needed to process, manage, and store large amounts of data which will be used by data scientists for analysis. They design the infrastructure and maintain data pipelines and warehouses to leverage data for strategic advantage.
Competences and skills
28 ESCO relationsEssential competences 1 competence
Occupation specific
0 competencesNo competences in this bucket.
Sector-specific
0 competencesNo competences in this bucket.
Cross-sector
0 competencesNo competences in this bucket.
Essential knowledge 8 competences
Occupation specific
1 competenceSector-specific
5 competencescloud technologies
The technologies which enable access to hardware, software, data and services through remote servers and software networks irrespective of their location and architecture.
digital
data models
The techniques and existing systems used for structuring data elements and showing relationships between them, as well as methods for interpreting the data structures and relationships.
digital
data storage
The physical and technical concepts of how digital data storage is organised in specific schemes both locally, such as hard-drives and random-access memories (RAM) and remotely, via network, internet or cloud.
digital
database management systems
The tools for creating, updating and managing databases, such as Oracle, MySQL and Microsoft SQL Server.
digital
unstructured data
The information that is not arranged in a pre-defined manner or does not have a pre-defined data model and is difficult to understand and find patterns in without using techniques such as data mining.
digital
Cross-sector
2 competencescomputer science
The scientific and practical study that deals with the foundations of information and computation, namely algorithms, data structures, programming, and data architecture. It deals with the practicability, structure and mechanisation of the methodical procedures that manage the acquisition, processing, and access to information.
digital
data analytics
The science of analysing and making decisions based on raw data collected from various sources. Includes knowledge of techniques using algorithms that derive insights or trends from that data to support decision-making processes.
digital
Essential skills and competences 14 competences
Occupation specific
1 competencedevelop data processing applications
Create a customised software for processing data by selecting and using the appropriate computer programming language in order for an ICT system to produce demanded output based on expected input.
digital
Sector-specific
5 competencesdesign database in the cloud
Apply design principles for an adaptive, elastic, automated, loosely coupled databases making use of cloud infrastructure. Aim to remove any single point of failure through distributed database design.
digital
establish data processes
Use ICT tools to apply mathematical, algorithmic or other data manipulation processes in order to create information.
digital
implement data warehousing techniques
Apply models and tools such as online analytical processing (OLAP) and Online transaction processing (OLTP), to integrate structured or unstructured data from sources, in order to create a central depository of historical and current data.
digital
manage data
Administer all types of data resources through their lifecycle by performing data profiling, parsing, standardisation, identity resolution, cleansing, enhancement and auditing. Ensure the data is fit for purpose, using specialised ICT tools to fulfil the data quality criteria.
digital
manage ICT data architecture
Oversee regulations and use ICT techniques to define the information systems architecture and to control data gathering, storing, consolidation, arrangement and usage in an organisation.
digital
Cross-sector
8 competencescreate data sets
Generate a collection of new or existing related data sets that are made up out of separate elements but can be manipulated as one unit.
digital
manage quantitative data
Gather, process and present quantitative data. Use the appropriate programs and methods for validating, organising and interpreting data.
digital
- Scope note
- Excludes .
manage research data
Produce and analyse scientific data originating from qualitative and quantitative research methods. Store and maintain the data in research databases. Support the re-use of scientific data and be familiar with open data management principles.
digitalresearch
perform dimensionality reduction
Reduce the number of variables or features for a dataset in machine learning algorithms through methods such as principal component analysis, matrix factorization, autoencoder methods, and others.
digital
process data
Enter information into a data storage and data retrieval system via processes such as scanning, manual keying or electronic data transfer in order to process large amounts of data.
digital
store digital data and systems
Use software tools to archive data by copying and backing them up, in order to ensure their integrity and to prevent data loss.
digital
use data processing techniques
Gather, process and analyse relevant data and information, properly store and update data and represent figures and data using charts and statistical diagrams.
digital
use databases
Use software tools for managing and organising data in a structured environment which consists of attributes, tables and relationships in order to query and modify the stored data.
digital
Optional knowledge 3 competences
Occupation specific
0 competencesNo competences in this bucket.
Sector-specific
2 competencesSAS Data Management
The computer program SAS Data Management is a tool for integration of information from multiple applications, created and maintained by organisations, into one consistent and transparent data structure, developed by the software company SAS.
digital
Teradata Database
The computer program Teradata Database is a tool for creating, updating and managing databases, developed by the software company Teradata Corporation.
digital
Cross-sector
1 competencestatistics
The study of statistical theory, methods and practices such as collection, organisation, analysis, interpretation and presentation of data. It deals with all aspects of data including the planning of data collection in terms of the design of surveys and experiments in order to forecast and plan work-related activities.
Optional skills and competences 2 competences
Occupation specific
0 competencesNo competences in this bucket.
Sector-specific
2 competencesanalyse pipeline database information
Retrieve and analyse different types of information extracted from the databases of pipelines companies. Analyse information such as risks, project management KPIs (key performance indicators), goods transportation times, and document back-up processes.
digital
create data models
Use specific techniques and methodologies to analyse the data requirements of an organisation's business processes in order to create models for these data, such as conceptual, logical and physical models. These models have a specific structure and format.
digital
Cross-sector
0 competencesNo competences in this bucket.