12 Factor App in NodeJS Part 0: Intro

12 Factor App in NodeJS Part 0: Intro

Building production ready and scalable apps from scratch

We have all heard that attachments are bad, well this applies to software development too. Your app may be too dependent on OS or Cloud Provider. Your libraries might only work on specific OS or your app must be too much dependent on cloud SDKs. Some OS are better suited to certain tasks than others and a policy change in some cloud providers can lead to huge expenses.

This leads to huge obstacles when you need to scale your app and can lead to huge expenses since moving out of a particular cloud or OS is a herculean task. Your App should be flexible enough to make sure you dodge such bullets easily deployments must be as smooth as possible to be able to ship more features faster and without any bugs with minimal intervention of any human resource.

App must be able to scale automatically up or down on demand which leads to savings on cloud bills on nonactive hours and better User experience by delivering faster responses at peak times.

That’s why engineers from Heroku came up with the 12 Factor app methodology which applies to web apps in general. These methodologies when applied will help scale your app easily, make your app platform agnostic and make deployments smoother by maintaining proper build, test and deploy stages and keeping minimal difference between dev and prod environment. You can easily deploy your apps on AWS, Azure, GCP, Heroku or even on on-premise servers

As mentioned in the name 12 Factor app is a list of 12 factors, I will list them here briefly and will write a detailed post with code implementation in NodeJS; ExpressJS + TS to be specific in later posts

I. Codebase

Track your codebase in a revision control system such as git. Why? It helps maintain multiple versions of an app in a single codebase

II. Dependencies

Language runtime and its Package manager should be the only prerequisite to run the app. Other dependencies should be declared explicitly in a dependency declaration manifest file and should be isolated i.e. no global packages can be used inside the app. This results in deterministic builds and thus makes it extremely simple to set dev and prod environments

III. Config

An app’s config is everything likely to vary between deploys (staging, production, developer environments, etc) which are credentials to external services such as Database(MySQL, Redis), Cloud Providers or External APIs like Twitter, dropbox, etc. Such configs should be stored in environment variables. This prevents the accidental leaking of sensitive information and reduces the hassle of updating such configs by just updating environment variables instead of manually editing files on server

IV. Backing Services

A backing service is any service the app consumes over the network as part of its normal operation. Examples include datastores (such as MySQL or MongoDB), messaging/queueing systems (such as RabbitMQ or Kafka), caching systems (such as Redis). The app should be able to swap out a local MySQL database with one managed by a third party (such as Amazon RDS) without any changes to the app’s code. This makes the app reliable and easier to scale since Resources can be easily attached to and detached from deploys at will. For example, if the app’s database is misbehaving due to a hardware issue, the app’s administrator might spin up a new database server restored from a recent backup. The current production database could be detached, and the new database attached — all without any code changes.

V. Build, Release, Run

The build Stage is when we are done with developing code and are confident for deployment. We thus run a build command that outputs an executable file that cannot be modified by us.

Release Stage is where we combine our build with config which makes the app able to run. These releases should be tagged with a unique ReleaseID.

Run Stage is where we run our release and make our app accessible to the outside world.

These stages should be strictly separated and any code change should trigger a new build, release and run stage. The Run stage should be as simple as possible, process or system restarts can easily rerun our run stage without requiring manual intervention. This also simplifies rolling back to the previous release when required

VI. Processes

Our App runs on processes. It can be a single process on the developer’s machine or multiple processes on production. This process must be stateless, all states should be stored in a storage solution. A temporary state can be stored in a caching solution like Redis and a permanent state in a Database like MySQL. The app should be able to handle processes or system restarts and recover to the right state. Also, since our app is running on multiple processes, the same user can be routed to different processes and its state information becomes inaccessible. Making our processes stateless makes our app reliable and ready to scale

VII. Port binding

Your service must be self-contained and be able to bind to a port to serve requests without relying on other software such as nginx to create a web-facing service. This must happen entirely in user space, that is, within the app’s code. [why]

VIII. Concurrency

In the twelve-factor app, processes are first-class citizens. Using this model, the developer can architect their app to handle diverse workloads by assigning each type of work to a process type. For example, HTTP requests may be handled by a web process, and long-running background tasks handled by a worker process. Since our apps are already running on stateless processes, fearless concurrency is easily achieved and our app can scale without making any changes to code.

IX. Disposability

Stateless processes are disposable, meaning they can be started or stopped at a moment’s notice. This facilitates fast elastic scaling and makes zero downtime deployments possible. Our app should take minimal time to startup and shutdown gracefully. Our app should also be prepared for sudden death. All these make sure our app is reliable, easy to deploy and scales elastically

X. Dev/Prod Parity

To prevent “This works on my machine”. This happens because developers by using adapter solutions use lightweight solutions on development machines and different solutions on prod. Some inconsistencies between these two can cause production to break which makes our app unreliable and thus damages business

XI. Logs

Our app should not worry about routing or storage of its output stream. It should not attempt to write to or manage log files. Instead, each running process writes its event stream, unbuffered, to stdout. It will be the responsibility of the execution environment on how it should handle stdout. In case the dev is developing on his local machine, the developer will view this stream in the foreground of their terminal.

In staging or production deploys, each process stream will be captured by the execution environment, collated together with all other streams from the app, and routed to one or more final destinations for viewing and long-term archival.

XII. Admin Processes

Admin processes such as running database migration or running some script to debug should be run in an environment that is identical to a target environment. Twelve-factor strongly favors languages that provide a REPL shell out of the box, and which make it easy to run one-off scripts. In a local deployment, developers invoke one-off admin processes by a direct shell command inside the app’s checkout directory. In a production deployment, developers can use SSH or other remote command execution mechanism provided by that deployment’s execution environment to run such a process.