Oracle Exadata, R, Unix, Control-M Scheduling, DataStage, Microstrategy
1 developer Data engineer
╋ Development of data modules loading 10 Terabytes of data for fast performance of SQL statements.
╋ 13 external events added to the main table to optimise forecasting model requests: Weather, events and festivals, end date and promotions information products, ....
Massive data processing to reconstruct 3 years of sales history with 150 attributes and metrics describing each product in all Coles shops in Australia.
Within the Coles data department, we are part of the team building a fast history and large table storing historical sales of each Coles item (product) and external events with 3 years of history. The aim is to forecast demand and optimise the supply chain. Each Coles shop in Australia has a range of items - all products in the shops. The data ingestion team works with the data scientist team to build a fast and massive historical master table. The data scientists train their artificial intelligence models with this table to predict customer demand in each shop.
╋ Build a master Oracle Exadata table to maximise batch processing and SQL statements and provide detailed data to form Coles AI forecast requests.
╋ Consolidate historical information for 50 million items - merged and curated from DWH data, company operational systems and external sources.
╋ Ingestion of external data sources: obtain from various interfaces the most useful localised information to enrich AI models (e.g. from the Bureau of Meteorology/BOM and for each Australian shop, the weather forecast data from the nearest BOM station).
Rahul Kumar Pandey