Idris Rampurawala articles - 23 total
Building a Scalable API Event Logger using Pub/Sub, and BigQuery
Learn how to design a scalable, event-based API logging system that captures request and response details, publishes them to Google Cloud Pub/Sub, and stores them in BigQuery for real-time analytics and insights. Ideal for building API observability pipelines beyond traditional structured logs.
Implementing a Redis-Based Task Queue with Configurable Concurrency
Learn how to build a lightweight, high-performance task queue using Redis. This guide walks through implementing a Redis-based queue system with configurable concurrency leveraging celery to process them efficiently. Perfect for developers looking for a cost-effective alternative to cloud-based queuing solutions.
Handling Duplicates in BigQuery
Are duplicate rows causing data discrepancies in your BigQuery? Learn how to efficiently handle duplicates in BigQuery with this post, saving you time and improving the accuracy of your analysis.
Underrated Skills of a Senior Software Engineer
Summarizing some of the underrated skills of a senior software engineer from an engineer's perspective.
Writing Custom Django Database Functions
Understanding different ways to customize Django's rich set of features based on our needs is beneficial. This post focuses on ways to customize Django database functions that provide more control in filtering queries with helpful examples.
Parallelize Processing a Large AWS S3 File
This post showcases the approach of processing a large S3 file into manageable chunks running in parallel using AWS S3 Select. This is the fastest and cheapest approach to process files in minutes.
Efficiently Streaming a Large AWS S3 File via S3 Select
Working with large data files is always a pain. This post focuses on streaming a large S3 file into manageable chunks without downloading it locally using AWS S3 Select. It not only reduces the I/O but also AWS costs. This approach does not require any external libraries for processing.
User Flow with dropouts using D3 Sankey in Angular 10
This post summarizes the steps to create a user flow/user journey chart showcasing dropouts using the D3 Sankey chart in Angular 10
Exporting Data To Excel and CSV in Angular
While working on a web application, there are various instances where we would allow our users to download the data into their specific formats. One such requirement is to allow them to export the data in a spreadsheet (excel) or a CSV file. Let's find out how we can create such functionality in Angular.
Flask Boilerplate - Structuring Flask App
Introducing Flask Boilerplate to quickly get started with production-grade flask application. It has some additional packages pre-configured for ease of development.
Creating a Search Filter in Angular
Wondering how to create a search filter in Angular without using any external libraries? Check out this post and also learn about pipes and directives along with it.
Share Your AWS S3 Private Content With Others, Without Making It Public
There are various instances where we want to share our S3 object with users temporarily or with some specific expiration time without the need to make our S3 bucket public. Want to know how it can be accomplished? Read this post to know more.
Setting up Python workspace in Visual Studio Code (vscode)
Whenever as a programmer we start a new project (in any language), we are required to set up an environment for our project to run. The environment may include things such as editor, packages supporting our project, language-specific linter and formatter, etc. A stable environment helps you to focus on the main aspects of implementation and leaving everything else with the setup to handle. Let's get started setting up a Python environment in Visual Studio Code editor.
Integrating Heatmap.js with Mouse-over click in Angular
Ever wondered how analytics tools create heatmaps? Well, here's good news. I have implemented heatmap in Angular 8 with the help of heatmap.js. Also, added a mouse-over click functionality.
Your work cannot be measured by the number of hours you sit in front of your laptop!
Have you come across a scenario where you are being tracked by the number of hours you sit in front of your desk to complete a task? Read along to understand the implications of such a toxic culture.
Deploying Django with Celery and Redis on Ubuntu
Deploying production-ready applications is always a tricky job. In this post, I have tried to explain how Celery enabled Django applications can be deployed to Ubuntu with Redis, Nginx, and Gunicorn.
Interviewing without Resume? Give it a thought
It's been a week now interviewing candidates without me taking a look into their resumes. Been a good...
Enabling ENA support on your AWS EC2
Enabling ENA support on your AWS EC2 with the aws-cli with 5 easy steps
Monitoring Memory and Disk Metrics for AWS EC2 Instances
This post will demonstrate how to produce and consume custom metrics for Amazon CloudWatch
Building Simple, Yet Powerful Product Integration Documentation
Ineffective product documentation can lead to a loss of customer base. Check out the process of creating the effective, simple and powerful product integration documentation.
Configuring SSH for git
Configuring SSH for git not only helps you ease out the authentication process but also helps you quickly remove authentication dependencies.
Django model relationships without running migration on existing database
There are use cases where we are working on a legacy system and are not allow to change/alter the DB constraints, but want to utilise the Djando ORM features as well. This post will help to create Django model relationship constraints without running migration on existing database.
Creating Django models of an existing DB
We often work on an existing database system (legacy system). In this article we will discuss about integrating Django with existing database