GDPLE is my Wordle-inspired US state economy guessing game that I wrote last year, but this October I did some work on both the frontend and backend that I thought warranted a blog post.
After reading Domain Modeling Made Functional I decided to re-write the GDPLE backend in F#. The player is shown a breakdown of a US state's GDP by sector and has five attempts to guess the state correctly. Initially, the backend was written in TypeScript and Node using NestJS, which is based off of Express but includes some useful features for dependency injection and decorators to mark endpoints. While it strikes a good balance between something like Spring and vanilla Express, NestJS was overkill for the GDPLE backend, and you can end up writing lots of boilerplate when using the library.
The prior NestJS controller class is shown below, and the new F# backend has the same endpoints. The frontend hits
POST /puzzle_session
to get a UUID for the player's attempt on that day's puzzle that is written to local storage. That UUID is included in
POST /guess
to submit a guess and
GET /answer/:id
if the player has exhausted their guesses before reaching the correct answer. The
GET /economy
provides the GDP breakdown of the mystery US state for the frontend to create a treemap visualisation of the mystery state economy.
@Controller
()
export
class
AppController
{
constructor
(
private
readonly
appService
:
AppService
)
{}
@Get
(
"/economy"
)
async
getTargetStateEconomy
()
:
Promise
<
StateEconomy
>
{
...
}
@Post
(
"/puzzle_session"
)
async
postPuzzleSession
()
:
Promise
<
IPuzzleSession
>
{
...
}
@Post
(
"/guess"
)
async
ostGuess
(
@Body
()
body
:
GuessSubmissionRequest
)
:
Promise
<
GuessSubmissionResponse
>
{
...
}
@Get
(
"/answer/:id"
)
async
getPuzzleAnswer
(
@Param
()
params
:
PuzzleAnswerRequest
)
:
Promise
<
PuzzleAnswerResponse
>
{
...
}
}
F# is a joy to write in, but because I was pretty directly re-writing TypeScript into F#, there are a couple of TypeScript language features that I missed. The US state economy data was represented in a JSON file, and being able to directly import the file with
import stateRecordList from "./UsStates"
was convenient. TypeScript union types made working with hierarchical economic data very succinct as shown by the
getTotalGdp
function below.
export
interface
NonLeafEconomyNode
{
gdpCategory
:
string
;
children
:
Array
<
NonLeafEconomyNode
|
LeafEconomyNode
>
;
}
export
interface
LeafEconomyNode
{
gdpCategory
:
string
;
gdp
:
number
;
}
getTotalGdp
(economy: NonLeafEconomyNode
|
LeafEconomyNode): number
{
if
(
"children"
in
economy)
{
return
economy
.
children
.
map
(
(
node
)
=>
this
.
getTotalGdp
(node))
.
reduce
(
(
prev
,
cur
)
=>
prev
+
cur
,
0
)
;
}
else
{
return
economy
.
gdp
;
}
}
The function takes advantage of TypeScript's permissiveness and
if ("children" in economy)
isn't the best way to distinguish between the two node types. It is possible to do something equivalent with more type safety in F# by replacing the direct JSON import with a JSON type provider class and using discriminated unions as shown below, but it ended up being unwieldy by introducing four separate types for the economy data. Because only the leaf nodes of the economy tree structure have GDP data, calculating total GDP means distinguishing between a leaf node that must return a GDP value and a non-leaf node where the GDP sum needs to be recursively called over all child nodes. I am sure there is a way to use option types more creatively and coerce the type provider into using them, but at first pass I wasn't able to make it work, and I wanted to get something up-and-running relatively quickly. Type providers are such a great F# language feature, so I hope I'll soon find the time to get these working properly.
let
getTotalGdp
(
economyNode
:
StateEconomies.Root
)
=
let rec
loop
(
node
:
Node
)
=
match
mode
with
|
Leaf leaf
->
leaf.Gdp
|
StateEconomy se
->
(
se.Children
)
|>
Array.map loop
|>
array.sum
|
OuterChild oc
->
oc.Children
|>
Array.map loop
|>
array.sum
|
MiddleChild mc
->
mc.Children
|>
Array.map loop
|>
array.sum
loop economyNode.StateEconomy
|>
Math.Round
|>
Convert.ToInt64
Another aspect of the backend that I want to return to pertains to the database. While I ended up using F# Dapper, I originally wanted to use either an SQL type provider or SqlHydra for better database related type checking while editing, but these required referencing an OS-specific
.dll
and I didn't want the hassle while moving between MacOS and Linux while writing the new backend. When I tried to decrement the
id
column of every row in the
target_states
table (represented as
puzzleAnswerTable
in Dapper), I faced a compiler error as shown below because I wasn't able to use the
puzzleAnswer
field in such a self-referential way inside the
update
statement. This meant I had to use the raw query instead, with even less type safety than what F# Dapper provides:
let
deleteObsoletePuzzleAnswers
(
dbConnection
:
DbConnection
)
=
// More elegant, but wouldn't compile
let
updatePuzzleAnswer
(
puzzleAnswer
:
PuzzleAnswer
)
=
{
id
=
puzzleAnswer.id
-
obsoletePuzzleAnswerCount
;
name
=
puzzleAnswer.name
;
gdp
=
puzzleAnswer.gdp
}
// error FS0039: The value or constructor 'puzzleAnswer' is not defined.
update
{
for
puzzleAnswer
in
puzzleAnswerTable
do
set
(
updatePuzzleAnswer puzzleAnswer
)
}
|>
ignore
// Cruder, but compiled
dbConnection.Execute
(
$
"UPDATE target_states SET id = id - {obsoletePuzzleAnswerCount}, updatedAt = CURRENT_TIMESTAMP"
)
|>
ignore
While working on the F# re-write I found out that GitHub allows for self-hosted CI/CD runners, and I wish I had found out about these sooner. All the CI/CD I had done in the past had been on GitLab CI/CD runners or Azure DevOps pipelines and I mistakenly thought that GitHub's equivalent - GitHub Actions - was a paid offering. This means that in the past, deploying one of my personal projects onto my VPS (virtual private server) meant either using
scp
or
git pull
before manually restarting some
systemd
services. Every once in a while I'd forget to restart NGINX, or I'd mess up file permissions by accidentally deploying as
root
and while this was a great way to stay on top of my system administration skills, it could get annoying and introduced more friction while trying to push out updates. My GitHub actions for the backend were straightforward to implement but not very sophisticated: I'm running the actions directly on the VPS hosting the backend. But the CI/CD Actions made my life much easier while addressing the bugs on the F# backend. Instead of having to SSH onto the VPS for every fix, I could push out small fixes over lunch.
The CI story was more interesting for the frontend: I kept
running out of memory
while running
vite build, which wasn't terribly surprising given the 1 GB of memory on the VPS:
<--- Last few GCs --->
[1044134:0x67be140] 48423 ms: Scavenge (reduce) 380.7 (391.3) -> 380.0 (391.6) MB, 1.79 / 0.00 ms (average mu = 0.210, current mu = 0.079) allocation failure;
[1044134:0x67be140] 49129 ms: Mark-Compact (reduce) 381.1 (391.6) -> 379.2 (391.6) MB, 697.59 / 0.05 ms (average mu = 0.262, current mu = 0.313) allocation failure; scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
The JavaScript bundle was relatively small at 445.79 kB/145.92 kB when gzip compressed. Luckily I was using Preact as a drop-in-replacement for React, which gzips down to 9.96 kB but my problems seemed to be with bundling Material UI. There were over 10,000 'module level directive' warnings associated with the
@mui
NPM package:
$
npx
vite
build
&
>
/tmp/build
$
cat
/tmp/build
|
grep
-c
'node_modules/@mui.*Module level directives cause errors when bundled'
10924
There has to be some way to reduce Node's memory footprint or work around these Material UI issues, but I wasn't able to do so and never liked the component library that much anyway. When an actual professional frontend engineer recommended I look into
React Aria
I had an excuse to replace the library entirely. It's a very simple frontend and I only ended up using the
Button,
Modal, and
Dialog
React Aria components. The most complicated part of the frontend is the autocomplete text input for submitting a US State name as a guess, and if I was using React then the
ComboBox
element would have worked great for this input. But I ended up using an
Autocomplete
element from the
Mantine
component library because of a possible bug in Preact.
I set up
another repository
with a side-by-side comparison between Preact and React to troubleshoot the bug. Both applications render only a
ComboBox
with a single
ListBoxItem
underneath it as a selectable option. When the Preact application is run, the following error appears in the developer console:
Uncaught TypeError: Cannot set property previousSibling of #<Node> which has only a getter
at $681cc3c98f569e39$export$b34a105447964f9f.appendChild (Document.ts:119:13)
at Object.insertBefore (portals.js:49:22)
The relevant section of
Document.ts
is shown below, this is part of React Aria used to build a variety of different components.
export
class
BaseNode
<
T
>
{
//...
appendChild
(
child
:
ElementNode
<
T
>
)
{
//...
if
(
this
.
lastChild)
{
this
.
lastChild
.
nextSibling
=
child
;
child
.
index
=
this
.
lastChild
.
index
+
1
;
child
.
previousSibling
=
this
.
lastChild
;
}
else
{
child
.
previousSibling
=
null
;
// Line 119 of Document.ts: error thrown here
child
.
index
=
0
;
}
//...
}
//...
}
The React application in that repository has no issue with rendering
ComboBox. No Preact errors are thrown if
ListItemBox
elements are left out, which of course defeats the purpose of having the
ComboBox
in the first place. This means that the problem is with Preact and
ListBoxItem
in particular. With respect to the error itself, in the code segment above the
child.constructor.name
is equal to
"HTMLUnkownElement"
for Preact. The rest of this section is more speculative - I know very little about Preact internals. However, in comparing two stack frames up from
BaseNode#appendChild, Preact is calling the method on a lower level of the virtual DOM hierarchy than React, and I don't think that this lower level exists.
The code segment below is taken two stack frames above
BaseNode#appendChild. For Preact,
parentVNode.type
and
parentVNode._dom
have values of
"item"
and
undefined
respectively, and the
_dom
property of a Preact
VNode
is 'The [first (for Fragments)] DOM child of a VNode'.
1
While the function call looks a little different for React,
child.constructor
and
child.node.type
are
[[FunctionLocation]] Document.ts:227
and
"item". While React is inserting an
"item"
virtual DOM element into the DOM, Preact looks to be inserting the first
child
of an
"item"
virtual DOM element, which is
undefined.
Because the text inside of the
ListBoxItem
is the lowest level in the component hierarchy, I assume is that the
"item"
DOM element is the content between the
ListBoxItem
tags, which is 'Aardvark' in my bug demonstration repository. Line 119 of
Document.ts
is only reached once in both the Preact and React applications, so this isn't a matter of React not yet reaching the lowest level of the component tree, and the
ListBoxItem
definition is
const ListBoxItem = createLeafComponent('item', function (props, forwardedRef, item) {...})
which likely explains the
"item"
in both function calls.
// preact: children.js:343
function
insert
(
parentVNode
,
oldDom
,
parentDom
)
{
//...
parentDom
.
insertBefore
(parentVNode
.
_dom
,
oldDom
||
null
)
;
//...
}
// react: react-dom-development.js:11069
function
appendChildToContainer
(
container
,
child
)
{
//...
parentNode
.
insertBefore
(child
,
container)
;
//...
}
There's a good chance that this isn't a true bug and is rather a tradeoff in
Portal
components that the Preact maintainers had to make, but in the coming days I'll post an issue in the GitHub repository for the project. If it's not a bug then I'll be curious to see what I got wrong here.
I would have preferred to use React Aria for the autocomplete over importing a second component library, but my VPS can build and deploy the frontend with GitHub actions without running out of memory. To my surprise, this frontend rewrite barely impacted the JavaScript bundle size. In comparing the repository before the F# rewrite at
ee865c8
with the current most recent commit
fa98105
as of writing, the bundle size shrank by less than 5% down to 423 kB. When building on an Apple Silicon M1 processor, the peak memory during the build also went down a modest amount from 47.56 MB to 45.04 MB. However, the build time went from 9.62 to 3.42 seconds, and the number of modules transformed from 12,473 to 2,716.
2
Instead of rewriting the frontend to build on one of the cheapest servers offered by Digital Ocean, I could have instead moved the action runners to a homelab server and done any number of things to deploy it onto the VPS - something I plan to do anyway. But what's the fun in that?
Values taken using the MacOS
time
command, averaged over three measurements↩︎