RTC
en nl
🌟Developing a simple public ToDo web application
[ Not yet finished ]
First app

Objective

In my previous post, I implemented several logical servers on my Proxmox home server machine:
“templ-ubuntu”, “home-devdepserver”, and “home-utilityserver”.
See Architecture of my home server .

I now want to develop an initial demonstration web application in Golang with a SQLite database,
using the central repository for source code storage.
The application then needs to be delivered as OCI container and stored in the central binary registry.

Next, an application execution server environment needs to be created on my home server.
The created containerized application and database needs to be deployed to this server environment.
For this, I’ll install “home-testappserver”, and “home-prodappserver” as logical servers.

Finally, I want to make this deployed web application publicly accessible via tunneling.
The application should be accessible via a link in the “Demo” page of my Cloudflare website.
In this post, I want to complete my “virtual data center” with communication middleware for this.
Cleaning up existing “home-pve” and “home-backupserver” will follow later.
After this post, we’ll have the following situation:

Servers

With this project, I’m fully implementing all my ideas for the public website and a small virtual datacenter.
In the next project, I want to start further developing my ideas regarding a modern software factory:
application development tools, testing, security, CI/CD, OCI & Kubernetes, and app/infrastructure monitoring.


Result

1 - Purchase and installation of laptop

Laptop

I bought my first laptop and installed Linux Mint as the OS on it.
I chose a refurbished business laptop, suitable for web development, at a budget price.
See Architecture of my laptop .
I used this Linux Laptop to test and learn all the new software.
In the future, I will switch completely from Windows desktops to only Linux desktops.

2 - Security of home network, servers, workstations and software

Laptop

Because I want to make applications on my home network accessible from the internet,
I need to better secure access to local resources.
I also grant visitors at my home access to my Wi-Fi network,
and I also want to prevent them from accessing local resources other than the internet.
I therefore expanded my access security, and this was done at various levels.

At the network level, I tried using firewalls on routers and VLANs (virtual LANs).
By providing separate virtual networks on 1 physical network,
I can keep internet visitors and Wi-Fi visitors at home separate from internal users.
However, I am using a router provided by my ISP that does not support VLANs,
and therefore have to wait for an upgrade from my ISP.
The firewall rules on my router currently allow all outgoing traffic to the internet,
but block all incoming traffic (I will need to make very minor changes to this later in this post).

At the machine/OS level, I have installed and configured firewalls for all machines.
On my Windows desktop, this is the built-in Windows firewall,
and on my Linux Mint laptop, this is the UFW firewall software.
These workstations allow all outgoing traffic but block all incoming traffic by default.
Proxmox was installed as the hypervisor OS on my NAS/home server, and it provides a hierarchy of firewalls.
Proxmox software-implements its own internal network on the machine with a virtual switch,
which connects all LXCs and VMs to the home network.
At the node level and each LXC/VM level, Proxmox provides a separate firewall that needs to be configured.

See “Network and Firewall organization” section in Architecture of my home server .

Security must also be provided for every accessible service/application.
Security via SSH keys has already been implemented for the SSH service and Gitea server software.
The desktop and laptop workstation therefore have my personal private SSH key,
and all servers have my personal public SSH key to grant easy access.
For self-developed applications that are publicly accessible,
I only provide secure HTTPS or VPN access after (password) authentication by the user.
More explanation regarding the chosen authentication software will follow later in this post.

3 - Developing minimal demo web applications in Go

Develop app

3.1 - Develop demo_hello application

First, I created a HelloWorld application “demo_hello” (with prefix “example-” in Gitea) in Golang.
For the implementation, the standard HTTP library and the templ library in Go were used.
I chose Golang as the programming language because it’s an easy high-performance language,
and ideally suited for running in containers/cloud environments.
This HelloWorld project will later to used to easily initiate development for all new Golang projects.
The repo contains the GitHub/Gite configurations, local Git configurations, and the Golang setup.
It also includes a simple makefile to simplify the compile, linting, and testing workflow through automation.
Other pipeline logic will be added later.
This way, a smooth development workflow was enforced right from the start.
See Cheatsheet for golang beginners for more information.

This and all subsequent application development repositories are publicly accessible in my GitHub / RobertTC32 account.

3.2 - Develop demo_todo application

Next, the “demo_todo” repo (with prefix “app-” in Gitea) was created with demo_hello repo as starter.
The repo structure was designed to contain not only Golang source code,
but also Golang test code, SQL script files, pipeline specifications, etc.
Next, I created an SQLite database with SQL scripts for creating DB objects and test data.
SQLite is the most widely used database in the world and runs on all operating systems.
It is an SQLite database that supports all ACID features (such as transactions).
It is free, highly performant (in-process), and easy to use/manage (all data is written to a single file).
Sqlite readers never block other readers or writers, and writers never block readers but do block other writers.
I will also use an SQLite database in Kubernetes in later projects (with horizontal scaling and backup),
and perform performance tests on it.

After creating the database, an initial version of the demo_todo web application was developed.
This only displayed the database contents as todo rows in a table on the screen.

3.3 - Deploy containerized web applications on application server

For both web applications, I created the smallest possible OCI image,
and then installed these on my test and production application servers.
Containerization and deployment were done manually,
but in subsequent posts we will automate this (ansible).

After booting up on the application servers, the web applications were tested within the home network.

4 - Making web application publicly accessible via tunneling

networking

4.1 - Make demo_todo application accessible

The ToDo web application is hosted on my home network and isn’t readily accessible over the internet.
To provide public access, I first learned a lot about networking and created an overview.
See Overview of network technology for more information.

In this overview you will find:

  • Network technologies
  • Network middleware to support websites and web applications
  • Products to access applications in your home network from the Internet
Access app

Under construction

I ultimately chose to use Cloudflare Tunneling,
because I already had a domain name and a (free) Cloudflare account.
All I needed to do was create and configure a Cloudflare connector on my LAN.
I used Docker to install the connector.
My ToDo application is now accessible via my public website on Cloudflare.
So in this section I installed the Cloudflare-Agent software on “home-testappserver” and “home-prodappserver”.

4.2 - Complete implementation of demo_todo application

For the web presentation, I used the already familiar tailwind CSS technology. See Cheatsheet for golang advanced for more information.

Finally, web screens were created to add or remove a todo line, or to change todo details.
For this, I used AlpineJS and DaisyUI as additional front-end technologies.
AlpineJS is a small JavaScript library that adds new attributes to existing HTML elements.
AlpineJS is a small JavaScript library that adds new attributes to existing HTML elements.
This allows code to be executed in the browser without JavaScript programming,
for example, for form validations or communication with backend server code.
DaisyUI is a UI component library based entirely on Tailwind CSS,
which allows you to create more beautiful web interfaces faster.

To access the database, I used the sqlc tool,
which generates the db-access code in Golang from a text file containing SQL and specifications.

4.3 - Add CDN to demo_todo application

Cloudflare’s CDN functionality can be used to retrieve static web resources for web applications.
This allows this to be done much more efficiently and on the edge.

CDN

Therefore, I used this CDN capability for the ToDo application, resulting in the following architecture:

Arch

This image was created with “Mermaid” software.
I used “Markdown Preview Mermaid Support” and “markdownlint” as VSCode extensions.
In previous projects, I already used “PlantUML” and “Drawio” in VSCode to create diagrams.
Mermaid can only create simpler diagrams than PlantUML,
but the renderer is already included in GitLab, GitHub, and Gitea.

5 - Make web applications publicly accessible without tunneling

5.1 - Install port forwarding and reverse proxy

ArchAlt

I will also later make self-hosted web applications public via https and port forwarding.
A client can then access my web application through my home WAN router,
which forwards the request via port forwarding to a reverse proxy (such as Nginx, Traefik, Caddy).
This reverse proxy will convert the https request to an http request,
and forward it to my self-hosted web application.
This does require a static public IP address, which could be resolved via DDNS.
It also requires a valid SSL certificate, which can be solved for free with the DNS Challenge.
The reverse proxy must support this.
This then results in the above architecture.

Next, I installed the Cloudflare-DDNS software on the Proxmox node,
and Traefik software alongside Cloudflare-Agent on “home-testappserver” and “home-prodappserver”.

5.2 - Make demo_hello application accessible

To test this, I made the “HelloWorld” application publicly available in this way.
I installed and used Traefik as a reverse proxy for this.
Because I want to work as securely as possible, I used Cloudflare to hide my own public IP address.
This way, I can also make applications temporarily available,
so that other developers can easily test them, for example.

Under construction

6 - Add access security to web application

6.1 - Install authentication and authorization middleware

Under construction

choice of “OAuth2/OIDC” (OpenID Connect) as Authentication and Authorization technology;
“Authelia” software (vs Microsoft ADFS, Keycloak, Authentik) is used as SSO implementation (with MFA support),
because it is open source and more lightweight.

install “Authelia” software alongside Cloudflare DDNS on “home-testappserver” and “home-prodappserver” Linux containers

6.2 - Complete implementation of demo_hello application

Multilingualism (nl, en) has been provided and the language to be used can be selected in the request URL.
Develop security logic and add it to the application and services.