Shipping log from various sources to AWS OpenSeach (P3)

byGinkMon, 22 Jul 2024

Based on the OpenSearch cluster created in previous posts, let's now see how to ship log data from various sources into this cluster. This will make it highly useful, functioning as a central logging system.

Setting up a log system with AWS OpenSearch (P2) - Google Authentication

byGinkThu, 02 May 2024

Continue the series of building a log system. Now we will check how to configure Google Authentication to let our users login OpenSearch Dashboard via Gmail or organization email (G-Suite).

Setting up a log system with AWS OpenSearch (P1)

byGinkTue, 30 Apr 2024

When we need a log system for applications deployed on AWS, we often think of CloudWatch as the first choice. However, after using CloudWatch for a while, I immediately noticed some limitations with this service.

  • It is difficult to search logs effectively, especially when there are hundreds of different log groups.

  • Not all developers have permission on AWS to open the CloudWatch console for accessing log data.

In case we want to build a decent log system, ELK seems to be a better choice. However, building and maintaining an ElasticSearch cluster is not an easy task. Log data will fill up every day and sooner or later we have to think about the way to increase nodes based on the size of the data. In this case, AWS OpenSearch seems to be a pretty good alternative. We will learn how to build a log system with this managed service in this article.

How to set up MongoDB with replica set via Docker Compose?

byGinkMon, 29 Jan 2024

MongoDB with replica set is a must if we want to work with features related to transactions. It's enabled by default if we're working with Mongo Atlas. But on local development environment, setting up a MongoDB with replica set can be a bit tricky. This short tutorial will guide you how to do it with Docker.

This article uses MongoDB 7.0, which is the latest version at the writing time.

Implement a middleware in Actix-web

byGinkWed, 29 Nov 2023

Middleware is an extremely useful tool for any web framework. It provides us a process to filter requests as well as route requests with similar characteristics. Actix-web is no exception. Let's explore how to create middlewares with this framework.

Common things we need as SysAdmins on Cloud

byGinkTue, 28 Nov 2023

The world of technology is gradually shifting from traditional servers to the Cloud. Consequently, administrators are being replaced by DevOps engineers. Traditional management tools now need to be installed using containers. And Alpine has become a useful tool for us, especially when accessing systems running containers such as Kubernetes, OpenStack, ECS, and others.

Below are some essential tools that can be installed in Alpine and are commonly used.

Recompile Nginx packaged by Apt on Ubuntu

byGinkSat, 22 Apr 2023

Sometimes we need a few extra features for Nginx and there's no other way but adding a new module. But getting the source from nginx repo can cause some conflicts and totally break the way we configured it, which is familiar by the prebuilt package from Apt.

Grab the source and configuration from existing Apt package and just modify a bit to add what we need is always better. Let's explore how to do it.

Key points to design application following DDD

byGinkSun, 26 Mar 2023

Just a take notes about Domain-Driven Design (DDD). Have you ever heard of it?

Best practice to deploy Golang with Docker

byGinkTue, 28 Feb 2023

Golang applications are ideal for Docker containers due to their single binary executable, but the base image chosen for building greatly affects the Docker image size. A heavy-weight golang base image from Docker Hub contains the OS filesystem with tools and packages, resulting in a very large docker image.

Alternatively, using multistage Dockerfile to build the binary combine with a minimalist base image like Alpine, busybox or scratch can create a much smaller image. As long as there are no cumbersome dependencies or filesystems, going with the smaller image size is typically advantageous.

Sound interesting? Let's check out the right way to build a very tiny executable for it.

Dealing with newline characters in environment variables

byGinkSun, 05 Feb 2023

An environment variable is a value that can be passed to a computer's operating system or an application at runtime. It provides a way to store configuration or sensitive information that can be used by multiple applications and scripts.

But sometimes, dealing with the variable that contains newline characters inside can be a headache. Especially when we need to convert between exported ENV and the infamous .env files. Let's see how we can do it correctly.

Multiple Databases with Active Record

byGinkWed, 10 Aug 2022

Ruby on Rails, often shortened to just Rails, is a popular framework that provides developers with a range of tools for building web applications.

One of its key features is the Active Record library, which provide a layer responsible for representing business data & logic, including a set of tools to work with database. However, many applications today require the use of multiple databases, especially with the replication model. In this article, we'll explore how to manage multiple databases in Ruby on Rails using the Active Record library.

How to test API in an Express application using Jest and SuperTest

byGinkSun, 06 Feb 2022

Testing is a very important part of software development. The goal of software testing is to find errors, gaps, or missing requirements in the implemented code compared to the actual requirements. The sooner you start testing, the better.

In this article, I'll show you how to write tests for your Express API with Jest and Supertest.

What's the difference between TDD and BDD?

byGinkSat, 05 Feb 2022

Test-Driven Development (TDD) and Behavior-Driven Development (BDD) are both software development methodologies that aim to improve the quality of code and reduce the number of bugs by using automated tests. While they share some similarities, TDD and BDD have distinct differences that we need to understand clearly before choosing which one to apply in the development flow.

Pulling images from AWS ECR to Kubernetes automatically

byGinkWed, 22 Sep 2021

ECR is a docker registry come from the giant AWS. It’s just like docker hub but fitting better with AWS ecosystem. If you choose ECR as your registry and deploy containers inside Amazon cloud, then it’s just a breeze. You have everything supported, just grant ECR permission with your IAM account and let it be done.

But if you’re deploying on your own K8s cluster or with another clouder like GCP, Azure, DigitalOcean… you’re out of luck, definitely. I’ve encountered the same issue when trying to deploy my product on DigitalOcean cluster with private images from ECR. Have to re-login each time rolling out the system is really annoying. So I have to find a way to get over it.

Understand labels and selectors in Kubernetes

byGinkFri, 10 Sep 2021

When working with Kubernetes (k8s), labels are the key to identify the objects. Most of configuration rely on labels to work correctly.

Basically, it’s meaningful to users but not imply semantics to the core system. And we often use labels to organize and select a subset of objects to apply configuration. Let’s see how to use label in the right way.

How to use Modules and Mixins in Ruby

byGinkSat, 10 Apr 2021

We already know that Ruby is an object-oriented programming (OOP) language. But the matter is it doesn't support multiple inheritance. Which means any class in Ruby can inherit from only one single parent class.

However, we still have several alternatives to achieve similar functionality. One of its key features is the ability to create and reuse code with modules and mixins. In this article, we'll explore what Modules and Mixins are, and how they can be used to organize and reuse code in a Ruby program.

Ruby blocks, procs & lambdas. What are they?

byGinkSun, 14 Mar 2021

Ruby is a high-level, dynamic programming language known for its simplicity and elegance. One of its core features is the ability to create and manipulate blocks, procs, and lambdas, which are all types of closures in Ruby. These closures are blocks of code that can be passed around as objects and executed at a later time, making them a powerful tool for implementing functional programming techniques in Ruby.

However, despite their similarities, there are some key differences between blocks, procs, and lambdas that you need to be aware of in order to use them effectively. In this article, we'll take a closer look at these differences and explain when and why you might choose one over the others.

Securely verify Facebook user from backend

byGinkSat, 02 Feb 2019

The common case when we implement the feature Sign-in with Facebook/Google is everything was done by client side.

If you just need a simple solution for faster login to application, skip all the hassle of registration back and fort between application and backend server. That’s quite enough. But if you need data integrity and more safety, there's probably more work to do on the backend to verify that.

The difference between javascript Array iteration methods

byGinkMon, 10 Oct 2016

Working with javascript Array is quite interesting because it provides you a lot of built-in methods. The long list can be started with forEach(), map(), filter(), find(), every() and so on. Each one has a different purpose and mostly cover all what you need. But sometimes, it can be a challenge for a newbie to understand which to use correctly.

Today, I’ll list out some commonly used methods and make it more clear for anyone new to it.


© 2016-2024  GinkCode.com