Skip to content

Open-source status page, incident response and uptime monitoring

✏️ (#211)

Thibault Le Ouayb12ff05703c9
diff --git a/apps/web/public/assets/posts/migration-planetscale-to-turso/latency.png b/apps/web/public/assets/posts/migration-planetscale-to-turso/latency.png
new file mode 100644
index 000000000..f5b44d15d
Binary files /dev/null and b/apps/web/public/assets/posts/migration-planetscale-to-turso/latency.png differ
diff --git a/apps/web/public/assets/posts/migration-planetscale-to-turso/planetscale-latency.png b/apps/web/public/assets/posts/migration-planetscale-to-turso/planetscale-latency.png
new file mode 100644
index 000000000..f01c210df
Binary files /dev/null and b/apps/web/public/assets/posts/migration-planetscale-to-turso/planetscale-latency.png differ
diff --git a/apps/web/public/assets/posts/migration-planetscale-to-turso/turso-latency.png b/apps/web/public/assets/posts/migration-planetscale-to-turso/turso-latency.png
new file mode 100644
index 000000000..7526c25c5
Binary files /dev/null and b/apps/web/public/assets/posts/migration-planetscale-to-turso/turso-latency.png differ
diff --git a/apps/web/src/app/blog/page.tsx b/apps/web/src/app/blog/page.tsx
index f58c8674c..e9e29b7f3 100644
--- a/apps/web/src/app/blog/page.tsx
+++ b/apps/web/src/app/blog/page.tsx
@@ -14,23 +14,29 @@ export default async function Post() {
<Shell>
<div className="grid gap-8">
<h1 className="text-foreground font-cal text-4xl">Blog</h1>
- <div>
+ <div className="space-y-4">
{posts.map((post) => (
- <Link href={`/blog/${post.slug}`} key={post.slug}>
- <section>
- <p className="text-foreground font-cal text-2xl">
- {post.title}
- </p>
- <p className="text-muted-foreground">{post.description}</p>
- <p className="text-muted-foreground mt-1 text-xs">
- {post.author.name}
- <span className="text-muted-foreground/70 mx-1">&bull;</span>
- {formatDate(new Date(post.publishedAt))}
- <span className="text-muted-foreground/70 mx-1">&bull;</span>
- {post.readingTime}
- </p>
- </section>
- </Link>
+ <div key={post.slug}>
+ <Link href={`/blog/${post.slug}`}>
+ <section>
+ <p className="text-foreground font-cal text-2xl">
+ {post.title}
+ </p>
+ <p className="text-muted-foreground">{post.description}</p>
+ <p className="text-muted-foreground mt-1 text-xs">
+ {post.author.name}
+ <span className="text-muted-foreground/70 mx-1">
+ &bull;
+ </span>
+ {formatDate(new Date(post.publishedAt))}
+ <span className="text-muted-foreground/70 mx-1">
+ &bull;
+ </span>
+ {post.readingTime}
+ </p>
+ </section>
+ </Link>
+ </div>
))}
</div>
</div>
diff --git a/apps/web/src/content/posts/migration-planetscale-to-turso.mdx b/apps/web/src/content/posts/migration-planetscale-to-turso.mdx
new file mode 100644
index 000000000..8c9bebd46
--- /dev/null
+++ b/apps/web/src/content/posts/migration-planetscale-to-turso.mdx
@@ -0,0 +1,209 @@
+---
+title: Why we migrated from PlanetScale to Turso.
+description:
+ Discover the reasons behind our decision to use Turso as our main database.
+author:
+ name: Thibault Le Ouay Ducasse
+ url: https://twitter.com/thibaultleouay
+publishedAt: 2023-08-20
+image: /assets/posts/migration-planetscale-to-turso/latency.png
+---
+
+## What are we building ? 🏗️
+
+We are building an open source status uptime monitoring with a beautiful status
+page. Our goal is to provide a fast service to our users.
+
+To achieve this, we are hosting it on the Vercel Edge Runtime, which requires an
+Edge Database to fully benefit from it.
+
+We also aim to simplify contributions to our project by eliminating the
+requirement for an account on an external service.
+
+Our current data tech stack is:
+
+- [Next 13](https://nextjs.org/)
+- [Drizzle ORM](https://orm.drizzle.team/)
+- [Tinybird](https://www.tinybird.co/)
+- [Turso](https://turso.tech/)
+
+### What is PlanetScale ? 🗃️
+
+[PlanetScale](https://planetscale.com/) is a cloud-native database platform.
+
+It is built on top of the open-source database system Vitess. PlanetScale offers
+a managed database service that allows developers to deploy, manage, and scale
+databases easily. In my opinion, the standout feature of PlanetScale is its
+migration management system.
+
+### What is Turso ? 🌍
+
+[Turso](https://turso.tech/) is a new database provider.
+
+They provide an edge database that enhances the developer experience of SQLite
+for your users. It enables extensive replication, allowing you to distribute
+your data to numerous locations where your users are located.
+
+### History of the project 📜
+
+When we began the project, we opted for PlanetScale due to its simplicity and
+cost-free database option. They also don't have any cold start which is super
+nice for us. Since we wanted to build fast, it was a good choice when we started
+the project. The database was just a URL. Their migration tools, ensuring zero
+downtime, were also valuable to us during rapid prototyping and frequent
+modifications.
+
+I came across Turso on X (formely known as Twitter) and became interested in
+trying it out. It seems promising as their value proposition is an edge
+database, which is exactly what we need. We plan to deploy the status page to
+the edge (Vercel Edge Runtime). Calling a database in a specific region would
+negate the benefits of Vercel Edge Runtime.
+
+## What drove our changes ? 🤔
+
+### Performance 🏎️
+
+Not all of our users are in `us-east-1`. Users who are not located in
+`us-east-1` should receive the same level of performance as those in that
+region.
+
+I was playing with this [Vercel tool](https://edge-data-latency.vercel.app/) to
+determine the most suitable database for our users. Among the options, Turso
+proved to be the fastest.
+
+![Turso latency](/assets/posts/migration-planetscale-to-turso/turso-latency.png)
+Turso is around `60ms` for 5 serial queries globally.
+![PlanetScale latency](/assets/posts/migration-planetscale-to-turso/planetscale-latency.png)
+While PlanetScale is around `450ms` for 5 serial queries in a different region.
+
+We considered using read replicas from PlanetScale, but I was also interested in
+trying out Turso.
+
+With Turso, you only need a single URL for all your replicas, eliminating the
+need to add this logic in your codebase. Additionally, Turso's scaler plan is
+more affordable, offering 6 replicas for $29, whereas PlanetScale only allows 3
+replicas.
+
+Turso offers 26 regions, while PlanetScale only offers 11 regions (their AWS
+regions).
+
+### Better DX for contributors 🧑‍💻
+
+We are building an open source status page, and we want to make it easy for
+everyone to contribute to it.
+
+Asking our users to create an account on a new service is a bit awkward. We
+could have asked for our user to use Docker and create a MySQL8 database while
+developing locally. However I don’t like Docker, it consumes too much memory on
+my laptop 😁.
+
+Or we use a file and SQLite, as it is the best option for everyone. And it’s
+what we can achieve with Turso: seamless local development.
+
+## How the migration went. 🚀
+
+### SQLite has less a less powerful type system than MySQL. 📦
+
+The migration process from MySQL to SQLite requires some changes. SQLite has
+fewer data types compared to MySQL. For example, DateTime doesn't exist in
+SQLite, so dates should be treated as integers.
+
+There are only five types in SQLite:
+
+- `INT`
+- `INTEGER`
+- `REAL`
+- `TEXT`
+- `BLOB`
+
+We use Drizzle for type handling and migration in our codebase, so we weren't
+significantly affected by this. Additionally, our experience with SQLite has
+taught us the importance of implementing additional checks in our code, rather
+than solely relying on the database engine.
+
+### Drizzle ORM Migration 😱
+
+We had to write additional SQL for migration when transitioning from Planetscale
+and/or Prisma. Initially, it was a bit frustrating because Drizzle could not
+generate all the necessary migration code.
+
+```
+/*
+ SQLite does not support "Drop not null from column" out of the box,
+ we do not generate automatic migration for that, so it has to be done manually
+ Please refer to: <https://www.techonthenet.com/sqlite/tables/alter_table.php>
+ <https://www.sqlite.org/lang_altertable.html>
+ <https://stackoverflow.com/questions/2083543/modify-a-columns-type-in-sqlite3>
+
+ Due to that we don't generate migration automatically
+ and it has to be done manually
+*/
+
+```
+
+After migrating a couple of times, you realize that they are mostly the same,
+and you need to follow these steps.
+
+1. Create new table
+2. Copy data into new table
+3. Drop old table
+4. Rename new table
+
+As it can been seen in this
+[migration](https://github.com/openstatusHQ/openstatus/blob/9f610d7cafb892774ceda533b3f21aaa4a4b3d5f/packages/db/drizzle/0003_glamorous_living_mummy.sql)
+when we released the incident:
+
+```sql
+ALTER TABLE `incident` RENAME TO `incident_old`;--> statement-breakpoint
+ALTER TABLE `incident_update` RENAME TO `incident_update_old`;--> statement-breakpoint
+
+DROP TABLE `incident_old`;--> statement-breakpoint
+DROP TABLE `incident_update_old`;--> statement-breakpoint
+
+CREATE TABLE `incident` (
+`id` integer PRIMARY KEY NOT NULL,
+`status` text(4) NOT NULL,
+`title` text(256) NOT NULL,
+`created_at` integer DEFAULT (strftime('%s', 'now')),
+`updated_at` integer DEFAULT (strftime('%s', 'now')),
+`workspace_id` integer NOT NULL,
+FOREIGN KEY (`workspace_id`) REFERENCES `workspace`(`id`) ON UPDATE no action ON DELETE cascade
+);
+--> statement-breakpoint
+
+CREATE TABLE `incident_update` (
+`id` integer PRIMARY KEY NOT NULL,
+`status` text(4) NOT NULL,
+`date` integer NOT NULL,
+`message` text NOT NULL,
+`created_at` integer DEFAULT (strftime('%s', 'now')),
+`updated_at` integer DEFAULT (strftime('%s', 'now')),
+`incident_id` integer NOT NULL,
+FOREIGN KEY (`incident_id`) REFERENCES `incident`(`id`) ON UPDATE no action ON DELETE cascade
+);
+--> statement-breakpoint
+
+```
+
+## The Turso Doc 📚
+
+The Turso Documentation has been the major challenge during the migration. We
+faced difficulties finding the specific information. We needed to make it work
+both locally and on Turso.
+
+However, we have DMed [Glauber Costa](https://twitter.com/glcst), the CEO of
+Turso, and they are aware of this issue. Hopefully, they will update the
+documentation soon. We believe that with comprehensive documentation and plenty
+of examples, Turso will be an even greater product.
+
+## The verdict 🍾
+
+We are pleased with the change, regardless of whether our users are interested
+in the stack we use. We genuinely enjoy using this stack. Turso is still an
+early product, but we are confident that it will improve over time. I strongly
+believe in its future.
+
+P.S. We are building OpenStatus in public, follow
+[me](https://www.twitter.com/thibaultleouay) or
+[Max](https://www.twitter.com/mxkaske) on Twitter/X if you want sneak peaks of
+the upcoming features - it's fun to watch! 🍿