Position title
Data Engineer
Description
A Data Engineer designs, constructs, and maintains the systems and architecture necessary for data generation, storage, and analysis. They develop and optimize databases, data pipelines, and ETL processes to ensure efficient data flow and accessibility. Data Engineers collaborate with data scientists and analysts to understand their requirements and provide them with the necessary infrastructure and tools for their work.
Responsibilities
- Design, develop, and maintain scalable data pipelines to extract, transform, and load (ETL) data from various sources into data lakes or data warehouses.
- Optimize data pipelines for performance, reliability, and scalability.
- Create and maintain data models to support analytics and reporting requirements.
- Implement data schema designs optimized for query performance and data retrieval.
- Integrate disparate data sources, including structured and unstructured data, to provide a unified view of the data.
- Implement data integration solutions to ensure consistency, accuracy, and timeliness of data.
- Develop and implement data quality checks and validation processes to ensure data integrity and accuracy.
- Identify and rectify data quality issues through data profiling, cleansing, and normalization techniques.
- Manage and administer databases, including performance tuning, optimization, and capacity planning.
- Monitor database health and implement backup and recovery strategies to ensure data availability and reliability.
- Configure and maintain infrastructure components such as servers, storage, and networking resources to support data processing and storage requirements.
- Implement and manage cloud-based data platforms and services (e.g., AWS, Azure, Google Cloud).
- Collaborate with cross-functional teams including data scientists, analysts, and software engineers to understand data requirements and deliver solutions.
- Communicate effectively with stakeholders to gather requirements, provide updates, and address issues related to data engineering projects.
- Stay updated with emerging technologies and trends in data engineering and analytics.
Qualifications
Bachelor's degree in Computer Science, Engineering, or related field
Hiring organization
TEKhops
Employment Type
Full-time
Job Location
Anoka, MN, USA
Remote work possible
Date posted
March 11, 2024
PDF Export
