We hear about the interesting uses of IoT in consumer products like toothbrushes, mirrors, refrigerators, cars all the time. That is a fairly new phenomenon (21st century). One of the initial applications of IoT has been to improve efficiency in the manufacturing process.
In the 20th century, IoT data was primarily stored in large databases (or historians) locally (within the factory) because of broadband limitations and data analytics was limited because of expensive compute infrastructure. This has drastically changed with the rise of cloud computing and improved broadband connectivity. Now factories collect the data from their equipment, do time-sensitive processing…
Elasticsearch is increasingly gaining popularity in search and log analytics use cases. In this blog, I will share my perspective on how to achieve multi-region replication in Elasticsearch.
If you have a large website that allows users to search the content (News, Online marketplace, etc.), then latency and throughput become the key requirements. One of the increasingly popular solutions for this is Elasticsearch.
You may want to set up elasticsearch in multiple regions to reduce latency. Here come the problems!!!
- Elasticsearch is not a multi-region service and you would need different clusters in different regions to cater to regional searches.
- …
DevOps is one of the most sought after skill in the tech world at present. This is the first of the series of blogs to help you demystify and learn the skill.
The engineers (and processes) that build a software or application are called Developers. The engineers (and processes) that package, deliver and monitor the software are part of Operations. Historically, both the roles were siloed as developers usually meant building the application and operations usually meant provisioning the hardware, deploying the application and monitoring the performance of the application.
Advent of cloud computing replaced the hardware provisioning with virtualization…
No one should lose their money. Especially, by not doing anything. Right? What if I told you that you are losing your money every day by not doing anything.
Let me explain to you how?
On Jan 21, 2013, a pack of 100g Amul butter cost around Rs 23. On Sep 5, 2021, the same pack costs Rs 45. This is a 95% increase in 8.5 years or an 8.21% increase compounded annually.
If you had Rs 23 on Jan 21, 2013, you could go to the market and buy 100g Amul butter. If you kept that money in your…
With Real-Time Anomaly Detection
Logs are the holy grail for a company to react to any technical issue, identify the root cause, and fix the issue for the customer. More and more companies are moving towards predicting the issue and fixing it beforehand so as to improve the customer experience and reduce their costs in reactively fixing it.
I once worked with a company that had to fly a technician to the customer to inspect the product, identify the malfunctioning part, fly back and take the new part again to the customer. …
AWS services that let you move the data from on-prem to AWS
While many of the new ideas (Startups, MegaCorps) are built exclusively on the cloud, many customers still need to move the data from their on-prem servers to the cloud both batch and streaming. In this blog, I will take you through the following AWS services that let you move the data from on-prem to AWS:
- AWS Database Migration Service
- AWS DataSync
- AWS Storage Gateway
- Amazon Kinesis Agent
- AWS Transfer Family
An Amazon Web Services Tip
How to use AWS Elasticseach for disaster recovery?
Two important parameters of any disaster recovery strategy are:
Recovery Point Objective (RPO) — Age of backup files to allow business continuity following an incident. In other words, the latest point in the past to which you must restore the systems.
Recovery Time Objective (RTO) — Duration of time within which the systems must be restored after a disaster for business continuity. In other words, the acceptable time in which you must restore the system to RPO.
E.g., f our RPO is 10 hours and RTO is 1 hour, and we take…
Computation => Oversimplified
Recently, a friend (non-technology background) asked me the difference between EC2 and Lambda. So, I gave him the following example oversimplifying the traditional computing, EC2, and Lambda.
Computing, in simple terms, is the availability of computing resources to execute complex tasks and applications. For example, you book a ride through Uber, a computing resource (Computer Server) somewhere in the world that completes your request for you.
You watch a movie on Netflix, a computing resource (Server) streams the movie to your computer somewhere in the world.
In the traditional world, companies forecast the number of users and then invest upfront…
People (primarily from non-computer science backgrounds) often ask me what is cloud computing. Here is cloud computing explained simply.
Imagine the world before electricity generation & distribution companies arrived and everyone can generate their own electricity. Businesses are generating their own electricity; startups and individuals are generating their own electricity. Let’s take the first example in detail:
Businesses generating their own electricity:
You are the owner of the business and operate two cotton mills. …
IoT Data Ingestion (Tags) and Analytics
The numeric tags collected from the sensors and edge devices are sent to AWS cloud through OPC-UA server and sitewise gateway. On AWS cloud, the data is aggregated, enriched and persisted for short-term, long-term and near real-time monitoring and analytics.
Here is a sample architecture for data ingestion and analytics and its description.
Akhil Jain
Sr. Solutions Architect (Big Data, IoT, ML) at Amazon Web Services | https://www.linkedin.com/in/akhiljain01/
