Article by Narender Kumar, Big Data & Cloud Lead Developer Overview Let’s assume we have a huge amount of data in our data lake related to sales. …

Article by Gautam Kumar, Big Data & Cloud Practice Lead Abstract This whitepaper focuses on architectural best practices to attain the most value for the least cost with the required level of performance and reliability when building cloud-native Data-Lake/big-data solutions. In particular, this whitepaper explains how you can maintain a robust…

Article by Nirali Gandhi, Big Data & Cloud Lead Developer Dicom Data Source for Apache Spark This Project provides a scalable spark-based mechanism to efficiently read Dicom Images in Spark-SQL Dataframe. One year ago, we came across a problem in the Healthcare Domain where reading a huge number of Dicom Images for analysis was involved in…

Article by Ansari Faheem, Big Data & Cloud Platform Lead Knox Introduction. Supported Apache Hadoop Services. Supported Apache Hadoop ecosystem UIs. Practical Part. Verification of Knox Setup. Validate your topology file. Validate your auth users. Make sure the below properties are set from Ambari. Accessing Hadoop Cluster via Knox Run…

Article by Dilip Kumar Khandelwal, Big Data & Cloud Developer I have recently passed the Exam AZ-303: Microsoft Azure Architect Technologies and Exam AZ-304: Microsoft Azure Architect Design. Progress The most significant change is increasing infrastructure-related questions to almost half of the exam questions and reducing the section dedicated to development…

Article by Nirali Gandhi, Big Data & Cloud Lead Developer Introduction Azure Logic Apps is a cloud service that helps you schedule, automate, and orchestrate tasks, business processes, and workflows when you need to integrate apps, data, systems, and services across enterprises or organizations. Logic Apps simplifies how you design and…

Article by Gautam Kumar, Big Data & Cloud Practice Lead Purpose Through this article, I would like to familiarize the readers with some of the basic concepts of Data Lake and also take them through the journey of various flavors of data lake implementations across the industry. …

Article by Ankit Sharma, Big Data and Cloud Senior Solution Architect Overview Under Data Management Platform architecture world, we are pretty much familiar with the term Data Warehouse, which is popular from the era of 80’s and Data Lake which gained a lot of attraction in the last decade starting from…

Article by Dilip Kumar Khandelwal, Big Data & Cloud Developer Koalas is an open-source project that provides a drop-in replacement for pandas, enabling efficient scaling to hundreds of worker nodes for everyday data science and machine learning. Pandas is a Python package commonly used among data scientists, but it does…