The database for the twodos app was dropped because of a misconfigured environment variable, causing existing users to be logged out of accounts and unable to access their data for a while.

The problem has been resolved since and data has been restored. You can access the service here on twodos.app website or download the client apps.

Cause

The Docker image used for the deployment of backend API had an extra environment variable NODE_ENVIRONMENT added. The objective was to make the Docker image a development instance by default. On production, environment variables would be configured to override the development/staging values.

There was a data layer code that reads this variable on app start up and purges all the data from the database.

The sequelize database being force created when the Node environment is development.
The sequelize database being force created when the Node environment is development.

This line was not gated properly and read a single variable to make such a big decision. The gate has been two-factored and an additional environment variable is now expected before the data can be purged making it difficult to enter this gate by mistake. Especially on production environments.

Impact

The misconfigured environment variable caused the database tables to be dropped with the data being truncated. This led all the existing users to lose the access to the application and the data that they had created.

This was detected first on staging environment and then was propagated to the production environment.

Resolution

The impact on existing users was minimal. The data is now recovered, and the application is performing normally.

To prevent this from happening again, the data layer environment gates have been two-factored to prevent single environments from purging the databases. Moving forward, to make such changes on a database, two variables need to be set that indicate the intention to remove or modify the data.

The data backup was used to rollback to an earlier version of the database.

Sincere apologies for the inconvenience caused.


The changes applied are part of a wider effort to make the application simpler, stable, and scalable on the UI-part. The changes in the API will make it possible to extend and improve:

  • Users and account management, to allow easier account creation, login, sharing, and access management for privacy reasons.
  • Lists and tasks management, to better expose the lists and tasks actions so that native platforms can compliment the data (add reminders, change the themes, etc.)
  • State management of the application to make the application faster.
  • Improved testing of the platform (API + app) to ensure the application performs and delivers what it aims to deliver.

We’ve received feedback from you all, especially around:

  • Emails and tokens not being delivered.
  • Application breaking (especially state management).
  • Lists and the tasks management UI being unclear.

We’re moving slowly toward releasing the application with all the improvements deployed a) first to our API and then b) a new version of the app on the client platforms.

Thank you!


Discover more from Afzaal Ahmad Zeeshan

Subscribe to get the latest posts sent to your email.

Discover more from Afzaal Ahmad Zeeshan

Subscribe now to keep reading and get access to the full archive.

Continue reading