By migrating the customer from Azure Data Lake Storage Gen1 to Azure Data Lake Storage Gen2, Pragmatic Works helped them see a 40% savings in storage per month, totaling over $28,000 monthly.
A large retail company was struggling to redesign their e-Commerce channels. The customer was working on an initiative to replace their antiquated systems and legacy hardware and software. Their former on-premises systems required maintenance and management, and there were concerns about reliability of the environment. They were looking to transform their entire business to provide better customer service both online and in-store, and a more complete and cohesive shopping experience overall.
The company wanted a partner to architect, design and develop a modern data workload and were in the process of building FionA, a large-scale modern approach to data management. They were also interested in Azure options for data streaming. One of the primary goals was assistance with improving real-time, general Business Intelligence (BI) and the reporting experience as part of the overall project.
As part of this process, the retail organization was moving their data, such as item prices and promotions, from the systems that drive the physical store checkout process to the systems that drive their website and on-line check out process. More specifically, they were addressing item-level retail data directly from the store system, including two separate store/POS systems integrated into a single organizational data mart for data science analysis. This involved sales, taxes, coupons, tender (credit/cash/reward/EBT) and other detailed information. They were conducting this migration through batch processing and at the end of each day, but the data transport and transformation mechanics were causing a delay to the website and on-line check out systems over 7% of the time. This was causing the on-line price to reflect the sale or promotion from the previous day in 7 out of 100 records.
Pragmatic Works was brought in as a premier Microsoft partner to develop a more efficient migration path and ensure near real-time performance from the customer’s data, as well as establish a viable option for Azure data streaming. To accomplish these goals, Pragmatic Works conducted an eight-week Azure Databricks Streaming Pilot, with an additional two-week follow-up engagement.
Working with the company’s internal team, Pragmatic Works reviewed existing Azure assets required for the Pilot and to deploy additional assets. We then coded a process to move the streaming XML messages from their core systems into a Delta Lake using Azure Databricks and Azure App Functions.
Pragmatic Works replaced the daily file-based batch processing with Azure Event Hubs and was able to provide near real-time data as opposed to the prior daily batch. This eliminated the issue of inaccurate promotional redemption from their website, saving them thousands of dollars per month.
Azure Databricks provides a flexible, powerful and scalable platform to perform ETL and analytic processing. It also has the added benefit of allowing the customer to work in a serverless environment, while scaling at will and using virtually any language.
By migrating from Azure Data Lake Storage Gen1 to Azure Data Lake Storage Gen2, they only had to pay for the storage they use. “We are expecting about a 40% savings in storage per month, totaling over $28,000 monthly, by moving to Gen2,” stated their Director of Data Solution Architecture.
Additionally, they gained the benefit of being able to store their data once, and access it multiple ways depending on their needs, giving them multi-protocol access. ADLS Gen2 also offers increased performance, data consistency, and granular security.
By choosing Microsoft technologies, the retail organization was able to process information faster and adapt to a changing market that is now requiring real-time data for consumers to be satisfied, and businesses to stay afloat. It is also critical to utilize this real-time data to make purchasing decisions. This engagement led to new monthly expenditures of: $1,000 in Azure Event Hubs, $10,000 in Azure Databricks and $5,000 in Azure Data Lake Gen2 in development. Their consumption increased 3x in production to $3,000 in Azure Event Hubs, $30,000 in Azure Databricks and $15,000 in Azure Event Hubs per month.
Pragmatic Works supports and advocates for Microsoft technologies to help our customers grow and achieve quicker and more reliable solutions. Using Microsoft cloud technologies allows our customers to achieve their desired outcomes and maintain an edge against their competition. Our strong relationship with Microsoft and ability to understand project scope and provide an all-encompassing plan made the customer comfortable moving forward with dramatic and critical changes.
Any retail organization can vastly benefit from a solution like the one we implemented with this customer. Real-time data has never been more critical than in the current world environment, where stay-at-home orders are issued world-wide and online shopping has seen a dramatic increase. E-commerce channels have experienced a 52% revenue growth rate between the fifth and eighth weeks of 2020, the period when COVID-19 began its rapid spread outside of Asia.
For Pragmatic Works, the four things that most commonly come up during technical evaluations include security concerns, technical function/speed, ability to deliver accurate results, and user adaptability/ease of use. By supporting Microsoft technologies, we are confident that in these areas of focus we will always rank higher than our competitors due to the high quality and reliability of Microsoft products, along with the unfailing support of our Microsoft partners. For this engagement, Pragmatic Works worked with several key Microsoft representatives, and the opportunity was initially discovered at an Enterprise co-sell day in Charlotte, NC.
Having a scalable ETL platform, able to produce real-time data, along with a flexible pricing plan through Azure Databricks and Azure Data Lake Storage Gen2 is the ideal solution for any retailer. It provides the option for rapid growth without the risk of obsolete data causing revenue loss, and the ability to expand with market demand and not commit to annual pricing with the world’s current market instability. The solution Pragmatic Works implemented has widereaching industry potential and has never been more valuable than it is now with urgent consumer needs and a rapidly growing demand for online fulfillment.
If you want to learn more about integrating Data Analytics in your organization, our experts can help. As a Microsoft Certified Partner, we can help your business gain valuable insights into your data. Contact us or click below to learn more about our Remote Services and fully customizable solutions.