Generated AI image by Microsoft Bing Image Creator

Introduction

If you’re like me, you’ve probably experienced the frustration of developing any AWS applications in the cloud especially such as lambda, cloudfront, S3, etc using tools like SAM, CloudFormation etc.. Now that I’ve been delving into the world of data engineering, AWS Glue jobs is one of the main AWS data engineering tooling offerings, the frustrations in having to push Glue jobs directly in the cloud—waiting for job runs, nitty gritty AWS Glue binaries setup, dealing with slow feedback loops, and watching your AWS bill creep up with every test iteration. I’ve been there, and it’s not fun.

That’s why I built this local development environment. After experimenting with different approaches, I’ve put together a Docker-based setup that lets me develop and test Glue jobs on my laptop before deploying to AWS. The best part? It includes all the real-world components I actually use in actual cloud envs: Kafka for streaming data, Iceberg for my data lake tables, and LocalStack to simulate AWS services.

In this guide, I’ll walk you through my setup step-by-step. Whether you’re working on batch ETL jobs or real-time streaming pipelines, this environment will save you time, money, and a lot of headaches.

Generated AI image by Microsoft Bing Image Creator

Introduction:

Since my new types of interesting engineering work have been mostly tied with backend and data engineering as of late, I’ve been earger to do a check in on what’s been happening lately in the web browser space and how much the world of web and front end development has changed since stepping away little over 18 months at the time of writing, particularly when working and manipulating with arrays of objects.

Sure enough, modern JavaScript and TypeScript, powered by the latest ECMAScript features, are loaded with powerful tools for working with arrays of objects. They enable concise, readable, and efficient code, making it simpler to manage, search, filter, and transform data structures. In this blog post, I’ll cover actionable tips and tricks to level up your expertise in handling arrays of objects, starting with some refresher basics up to something more advanced takes.

Generated AI image by Microsoft Bing Image Creator

Introduction:

First off - have a Happy New Year of 2025 everyone! :)

What a busy and challenging year of 2024 it’s been.

Looking back, I’ve been wondering what can I look forward in uplifting my personal productivity when in comes to staying lazer-focus, maintain strong context aware of code repositories without losing to context switching, which I have been struggling to achieve for some considerable time.

Especially when it comes to having the decent monitor screen size as my external display to check out code, read code, write code, submit PRs, review PRs etc.

Hence the purpose of this writing up this blog post - describing my recent experience of installing ultra-wide curved monitor for my WFH setup.

Generated AI image by Microsoft Bing Image Creator

In the early days of my software development career, I relied on Heroku to host all of my personal apps without incurring any costs. However, over the years, Heroku has gradually shifted from a free model to a subscription-based service. As a software developer, I’m always on the lookout for platforms that offer better performance, scalability, and ease of use for hosting my applications while continuing to manage my projects on budget. After careful consideration and evaluation, I’ve decided to migrate all of my personal apps from Heroku to Render. In this guide, I’ll walk you through the migration process and explain the reasons behind my decision.

Generated AI image by Microsoft Bing Image Creator

Introduction

In my previous post, I mentioned the usefulness of apply software patterns for scalable Javascript applications so far. I thought of revisiting this again because after working as a software engineer/developer for a while, I always found myself that design patterns are heavily applied across all projects (outside of Javascript domain) I have seen. Those projects were anything around Java, PHP, Python, C++, etc, thus I found there’s a common notion that design patterns are ubiquitous when applying to solve interesting and challenging software problems in our modern times.

Generated AI image by Microsoft Bing Image Creator

Introduction

Happy Belated New Year of 2024!

With the arrival of my shiny new MacBook Pro M2, I will explain In this guide how I setup my machine for a seamless software development experience.

I would love to keep my current Macbook Pro Intel Core i5 2017 edition to continue using it.. But, over the years, the CPU and memory requirements have gone up for more modern full-stack development using Docker, updated Chrome extensions, or upgrade browsers, my limited 8GB RAM and 250GB of SSD storage started to feel small for me to do any decent software development tasks these days. If I knew Apple is incredibly stringent to make memory and storage computing parts non-upgradeable after the purchase is made, I could have turned back the time to request upfront upgrade for these back then, and not to feel the terrible waiting grind for the systems resources to free itself whenever I start up Chrome browser or Docker background process, which ended up to seem like an eternal wait… ⏰⏱

So, forcibly speaking, I have to start moving from my old mac, and say hello to my new Macbook Pro M2!

Generated AI image by Microsoft Bing Image Creator

Before I start making my blog post intro, I have to say, Wow! It’s been a while since I made my post here. It’s almost the entire year of 2023 that’s about to draw its last curtain before 2024 is coming around the corner.

Since my last post earlier in the year, I mentioned my wedding plans mid-year of 2023 and now that’s all done and dusted. Naturally, the next thing to do after my wedding was to book our honeymoon to tour all the fun places around Malaysia and Indonesia towards the end of October 2023.

Here’s one of the best photo trips we took during our initial stay in KL.

The magnificent sight of Petronas Twin Towers during the night.

Generated AI image by Microsoft Bing Image Creator

Happy New Year! Here’s to my first blog post of 2023.

Since my last post, I have finally managed to get a number of errands to accomplish before the end of 2022. One of the most important items was getting my father’s ashes finally sent homebound back to Malaysia in September for his eternal rest. It was great to be back home and get a good holiday stint around my hometown of Sibu, Sarawak as well as the capital city, Kuching where I haven’t been for a long time to just wind down and relax, along with the wedding preparations underway downunder.

Now, that’s all out of the way, it’s time to get back into my world of blogging once more!

Last year, I faced a deceptively complex but interesting solution design problem when working with NodeJS event-driven programming model, EventEmitter and how we should go about writing up appropriate unit testing when listeners were subscribing to certain events being emitted.

Generated AI image by Microsoft Bing Image Creator

We’re edging towards halfway of 2022.. And boy. Look how time has gotten passed by.

I must admit I haven’t had the chance to sit down and write up another good tech blog post. A lot is happening in motion since the start of the year 2022 from my recent engagement to my fiancee of 3 and half years; busy with my day job as a contractor for a bank; finally getting my passport renewal done in Canberra (after almost 2 years of nationwide lockdown) as part of my travel plans to Malaysia so I can send my father’s ashes for his final resting place; along with talking plans having my fiancee and me moving together later in the year and preparing my next career technical certifications to take on, etc…

The list could go on.

Nevertheless, still, I’m glad to take the time to write about what I have accomplished recently.

That is…

I’ve decided to migrate my Octopress site to Hexo! 📦🚚

Generated AI image by Microsoft Bing Image Creator

If you’re like me, and you’ve built several Github repositories over time as part of developer learning fun projects, there’s probably a good chance these same repositories will not get maintained on some form of regular basis.

The reasons for this to happen can be wide-ranging - everything from being too busy at work, changing interests for different software frameworks, lifestyle priorities changes, changing career responsibilities etc, etc - there’s too many to list here.

They took the best of our coding life as months or years go by.

As a consequence, your Github repositories do become quickly stale over time.

This ‘staleness’ - come with old dependencies that do not get updated, which at worse, could be leaking software security issues over time.

I have over 50 repositories in my Github profile and there’s no way that I could keep track of knowing which repos have more security holes to address after another, let alone have to fix up security vulnerabilities dependencies one PR at a time on my own… 😨

Thus I need to find a way to auto-manage all these repositories without lifting a finger (much)

Without further ado, I found this useful toolchain from Github marketplace - Snyk.