F# vs TypeScript - Sorting Performance (Round 2)
Date: 2023-08-23 | create | tech | fsharp | typescript | performance |
In the past few posts, we've been exploring programming language performance benchmarks wrt sorting large amounts of elements:
- How fast can we sort 40k elements in TypeScript?
- How fast can we sort 1 Million elements in F#?
- F# vs TypeScript performance - Sorting 1 million elements
An observant viewer pointed out a problem with the benchmark comparison - the data set used for TypeScript's benchmarks differed from those used for F#'s, likely leading to a penalty for TypeScript.
In this post we'll try to fix this mismatch and answer the question:
Q: How does F# vs TypeScript performance compare, after fixing the dataset discrepancy?
Answer
When we fix the random number range discrepancy (limiting to int32), TS performs 4-8x faster for sorting primitive numbers.
However, the overall comparisons between F# and TypeScript remain largely the same with F# being about as fast or faster in each category - often by a margin of 1.5 - 4x.
Problems with the TypeScript benchmark
Thank you to user-kw9sv30c7f for the insightful YouTube comment pointing out this issue!
The problem with the previous iteration of the comparison was that the datasets each language was sorting differed.
- TypeScript: Semirandom values using
Number.MAX_VALUE(0, 2^1024 - 1) - F#: Semirandom values using
Int32.MaxValue(0, 2^31 - 1)
This means that TypeScript would have a much wider range of values to deal with - many of them being recognizable as numbers but extremely large numbers. Intuitively this means that TypeScript may need to do more work to sort these values and certainly means that the benchmarks have an obvious discrepancy in them.
A few potential ways this could negatively impact the TypeScript benchmarks as mentioned by user-kw9sv30c7f:
- TS random values are less likely to have "pre-sorted" segments in them -> more work
- TS value lookups in Node may be slower since "numbers greater than 2^31 - 1 are heap allocated, as opposed to stack or register allocated"
The fix for this was to limit the random value ranges to match across benchmarks. For this, I decided we should just use F#'s Int32.MaxValue as it's sufficiently large to get a decent distribution while remaining small enough to potentially capture any optimizations each language / runtime may have.
Updated TypeScript Benchmark Results
With random values constrained to match F#, the TS benchmark performs much better in the primitive number sorting category by about 4-8x. Other data types perform about the same.
TypeScript Sorting Performance (Round 2)
Result highlights reported as:
- ITEM_COUNT : TIME_MS
Previous TS Item Sort (primitive numbers, p90)
- 10: 0
- 100: 0
- 1000: 2
- 10000: 69
- 40000: 293
- 100000: 894
- 1000000: 13795
New TS Item Sort (primitive numbers constrained, p90)
- 10: 0
- 100: 0
- 1000: 1
- 10000: 11
- 40000: 78
- 100000: 176
- 1000000: 1667
Updated F# vs TypeScript Benchmark Comparisons
F# still pulls out ahead of TypeScript in most categories with number sorting performing a bit worse compared to the previous comparison in both primitive numbers and number properties.
(% represent how much faster F# is than TypeScript)
Previous F# vs TypeScript Performance Comparison
These values were calculated as part of the previous F# vs TS performance comparison.
F# vs TypeScript Performance (Round 2)
F# Raw Results
One consistent anomaly in each comparison is TS's primitive string sort. Across most benchmarks it outperforms F#'s by a substantial margin (5-45%). I'm not sure why this is but my guess is it has to do with the underlying datastructures w TS (and by extension JS) biasing towards perf and F# biasing towards copyability (due to immutable-first leading to lots of copies) but this is mostly a guess.
Next Steps
As evidenced here, all benchmarks have asterisks and often it's hard to figure out what those might be. You can find the latest code each benchmark runs on Replit:
For more performance benchmark explorations (including their asterisks) you might like:
Want more like this?
The best / easiest way to support my work is by subscribing for future updates and sharing with your network.