Science Publications

Sign Up
Sign In
Submit
Current Research in Public Health
Cite This Article Share Download PDF XML
  1. Home
  2. Journals
  3. Current Research in Public Health
  4. Archive
  5. Volume 1, Issue 1, 2020
  6. Article
Article Open Access November 24, 2022

Bridging Traditional ETL Pipelines with AI Enhanced Data Workflows: Foundations of Intelligent Automation in Data Engineering

Kushvanth Chowdary Nagabhyru 1,*
1
Data Engineer, USA
Publihed in: Current Research in Public Health (Volume 1, Issue 1, 2020)
Page(s): 82-93
DOI: 10.31586/ojes.2022.1345
Received
September 16, 2022
Revised
October 29, 2022
Accepted
November 20, 2022
Published
November 24, 2022
Keywords
Extract; Transform; Load; ETL; AI; Artificial Intelligence; Machine Learning; MLOps; Data Pipeline; Data Workflow; Data Engineering; Data Engineering Automation; Data Engineering AI; Intelligent Automation; Data Quality; Data Governance; Costs; Case Studies; Design; Future; Trends; Challenges; OpenAI; GPT-3; Selfie; OpenAI Codex; Large Language Models; LLM; Large Language Model
Creative Commons

This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright: Copyright © The Author(s), 2020. Published by Scientific Publications
Article metrics
Views
78
Downloads
15

Cite This Article

APA Style
Nagabhyru, K. C. (2020). Bridging Traditional ETL Pipelines with AI Enhanced Data Workflows: Foundations of Intelligent Automation in Data Engineering. Current Research in Public Health, 1(1), 82-93. https://doi.org/10.31586/ojes.2022.1345
ACS Style
Nagabhyru, K. C. Bridging Traditional ETL Pipelines with AI Enhanced Data Workflows: Foundations of Intelligent Automation in Data Engineering. Current Research in Public Health 2020 1(1), 82-93. https://doi.org/10.31586/ojes.2022.1345
Chicago/Turabian Style
Nagabhyru, Kushvanth Chowdary. 2020. "Bridging Traditional ETL Pipelines with AI Enhanced Data Workflows: Foundations of Intelligent Automation in Data Engineering". Current Research in Public Health 1, no. 1: 82-93. https://doi.org/10.31586/ojes.2022.1345
AMA Style
Nagabhyru KC. Bridging Traditional ETL Pipelines with AI Enhanced Data Workflows: Foundations of Intelligent Automation in Data Engineering. Current Research in Public Health. 2020; 1(1):82-93. https://doi.org/10.31586/ojes.2022.1345 
@Article{crph1345,
AUTHOR = {Nagabhyru, Kushvanth Chowdary},
TITLE = {Bridging Traditional ETL Pipelines with AI Enhanced Data Workflows: Foundations of Intelligent Automation in Data Engineering},
JOURNAL = {Current Research in Public Health},
VOLUME = {1},
YEAR = {2020},
NUMBER = {1},
PAGES = {82-93},
URL = {https://www.scipublications.com/journal/index.php/OJES/article/view/1345},
ISSN = {2831-5162},
DOI = {10.31586/ojes.2022.1345},
ABSTRACT = {Machine Learning (ML) and Artificial Intelligence (AI) are having an increasingly transformative impact on all industries and are already used in many mission-critical use cases in production, bringing considerable value. Data engineering, which combines ETL pipelines with other workflows managing data and machine learning operations, is also significantly impacted. The Intelligent Data Engineering and Automation framework offers the groundwork for intelligent automation processes. However, ML/AI are not the only disruptive forces; new Big Data technologies inspired by Web2.0 companies are also reshaping the Internet. Companies having the largest Big Data footprints not only provide applications with a Big Data operational model but also source their competitive advantage from data in the form of AI services and, consequently, impact the cost/performance equilibrium of ETL pipelines. All these technologies and reasons help explain why the traditional ETL pipeline design should adapt to current and emerging technologies and may be enhanced through artificial intelligence.},
}
%0 Journal Article
%A Nagabhyru, Kushvanth Chowdary
%D 2020
%J Current Research in Public Health

%@ 2831-5162
%V 1
%N 1
%P 82-93

%T Bridging Traditional ETL Pipelines with AI Enhanced Data Workflows: Foundations of Intelligent Automation in Data Engineering
%M doi:10.31586/ojes.2022.1345
%U https://www.scipublications.com/journal/index.php/OJES/article/view/1345

TY  - JOUR
AU  - Nagabhyru, Kushvanth Chowdary
TI  - Bridging Traditional ETL Pipelines with AI Enhanced Data Workflows: Foundations of Intelligent Automation in Data Engineering
T2  - Current Research in Public Health
PY  - 2020
VL  - 1
IS  - 1
SN  - 2831-5162
SP  - 82
EP  - 93
UR  - https://www.scipublications.com/journal/index.php/OJES/article/view/1345
AB  - Machine Learning (ML) and Artificial Intelligence (AI) are having an increasingly transformative impact on all industries and are already used in many mission-critical use cases in production, bringing considerable value. Data engineering, which combines ETL pipelines with other workflows managing data and machine learning operations, is also significantly impacted. The Intelligent Data Engineering and Automation framework offers the groundwork for intelligent automation processes. However, ML/AI are not the only disruptive forces; new Big Data technologies inspired by Web2.0 companies are also reshaping the Internet. Companies having the largest Big Data footprints not only provide applications with a Big Data operational model but also source their competitive advantage from data in the form of AI services and, consequently, impact the cost/performance equilibrium of ETL pipelines. All these technologies and reasons help explain why the traditional ETL pipeline design should adapt to current and emerging technologies and may be enhanced through artificial intelligence.
DO  - Bridging Traditional ETL Pipelines with AI Enhanced Data Workflows: Foundations of Intelligent Automation in Data Engineering
TI  - 10.31586/ojes.2022.1345
ER  -