Testers and Opinions wanted

Internet Connectivity
1

Ok, so I’ve set up a forum demo at the following address;

It’s been tweaked slightly to allow only logged in users, primarily to prevent search engines from trying to index it. (which we don’t want)

What is it?

It’s a copy of the main forum site, literally a restore of the main backup from this morning.

What’s it running on?

A raspberry Pi5.

Where is it running?

My desk (on the end of a broadband connection), in “docker” (because that’s the default discourse mechanism) which in turn is running inside a standard LXD style container. (presented to the net via a Zero Trust tunnel)

And the point is?

Well here is the comparison;

  • Cost, hosted version is £15 a month, desktop version is £0.
  • Scale, hosted version has 2G of RAM and 1 Core, desktop has 8G and 4 Cores

To Help

If anyone can spare a few minutes to log in, I’d be interested to know how you find it in terms of speed of response and any issues. To me it seems fractionally quicker even as a single user. Given much of the page delivery process will be running on a single core, the main performance benefits would be coming from multiple users in parallel.

Perceived performance

Ok, so it’s really hard to know how to benchmark this, but as far as I can see the caching is getting in the way of most of the performance tests.

  • Live copy on Digital Ocean, 78 requests per second
  • Desktop copy on RPi5, 74 requests per second

That said (!) to me the RPi feels a little quicker, despite the connection running to my desktop over a VPN tunnel, an overhead the current live copy doesn’t have.

Extra tech

RPi5 (host) is running ZFS for storage on a 128Gb SSD.

$ zpool list
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
pool   102G  4.78G  97.2G        -         -     0%     4%  1.00x    ONLINE  -

It’s tuned to make best use of memory and processors (obviously it can run in much less memory);

Screenshot from 2024-04-14 14-40-50
Screenshot from 2024-04-14 14-40-50975×587 93.9 KB

One of the nice features I discovered yesterday is that LXD (or INCUS as it’s now known) is able to move containers between Raspberry Pi’s relatively quickly, so reorganising hosted services is relatively painless … and ZFS gives checkpoints and incremental remote backups, which is something Digital Ocean don’t seem to offer, at least not as yet.

2

I agree that it feels fractionally quicker but not measurably so, definitely not slower.

3

I replied on the other site:

The speed seems to be about the same as the normal site - certainly quicker than my brain.
The advantages that you listed seem to be very well worth having.

K

4

Yes, the only downsides I can see are that it’s subject to;

  • Local power cuts
  • Broadband failure

We rarely get any power outages here (over the last 20 years) and when we do it’s only for very short periods, so I’m looking to put in some UPS gear to mitigate that.

Broadband, has historically been pretty good subject to odd copper-related faults now and again. Any failures on the fibre would likely be catastrophic only, so not likely to happen with any frequency. I’m not sure outages on the fibre should be any more likely than in the Data Center, and we have had a few issues there.

Interestingly Virgin media have just dug up our road (much to the displeasure of many locals, understandably if you look at the quality of some of their remedial work) and there is a tap-off 12 feet from my BT fibre delivery point … so there would seem to be scope for a second / backup fibre in future. I guess this is the only good reason I can think of for Virgin “not” just reselling BT fibre …

My only niggle is that the location of my kit is different to the location of the fibre, so unless I reorganise I’m going to need two lots of UPS :frowning:

5

if it helps regards UPS, I found that most/any battery bank charger works as a basic UPS, but I have upgraded to something like for my router (which is away from my main server).

Just check the power needed and provided :wink:

https://www.amazon.co.uk/Konnected-UPS-Backup-Battery-Raspberry-White/dp/B08JTVY87T/

…being just one of many on the market…

6

Mm, that’s the kind of thing. What I’ve been looking at today is two things;

  • A power source (something like the link)
  • A power distribution mechanism (multi-port USB hub)

So essentially I can power a bunch of Pi’s via a battery off one wall plug. For this to work I reckon 20W per outlet should be reasonable for my usage, so a 5-port needs to be at least 100W.

So ideally it needs to be a battery with something like 100-200W/h which should be fairly minimal, but I then need 4A per USB port. This seems to be a sticking point as most seem to be 2-3A. Still looking :slight_smile:

7

I agree with Gaz511 100%, if anything slightly faster but barely measurable

8

:sunglasses:

Ok, this sounds workable. In which case what I’m going to do (subject to other time pressures) it try to get hold of a dedicated RPi5 setup and migrate the Forums onto it, doesn’t sound like there are any performance penalties and running 4x the CPU cores and 4x the Memory can’t hurt :slight_smile:

Setup I’m currently looking at it;

  • 100W USB A or C type 5-port switch (so up to 20W/unit)
  • Fed from current HP ProCurve switch
  • RPi5 with Active Cooler
  • NVMe Hat
  • 256G M.2 SSD

The M.2 drive should be ~ 2.5x the current IO speed. (currently running of an old external USB SSD) … I actually have 3 other containers now running off the machine currently running the test forums, doesn’t seem to have any impact on performance :slight_smile:

I’m using the “incus copy” command to move the instance between RPi hosts, which is relatively quick, if there are any problems then technically I should be able to set up a new INCUS host on Digital Ocean and just copy the entire container as-is back onto the cloud.

Running things like Discourse Forums as portable services that can be copied in and out of the cloud relatively transparently, does seem to be quite an attractive way forward. Maybe not so good for hosting companies who are fans of vendor lock-in, but for everyone else … :wink:

9

MP,

I can’t find any USB charges which provide 5v 5a… But let me know how your hunting goes, as I may follow suit. …But I fear we may need dedicated plugs (or POE) - boo-sucks!

also, do you intend to use PI-OS? I’m torn between that and Alpine (which I use for my current docker VMs). … Or like your other post, are you heading towards a pure Debian build??

10

Ok, so I had the same problem as you which means I now have a standard 27v unit sat on my desk. I’ll probably experiment with something when I have more time, there are some 100W-160W 6-port units out there which would probably work, but they all seem to be 3xUSBC and 3xUSBA … and if I’m going down that route I’d like consistency.

POE is notoriously problematic, something I’d rather avoid, not least as it means a POE hat which means you can’t use a standard case. I’ve just finished testing a new Argon NEO case which has a heatsink and it’s own fan. Tops out at 61 degrees which is pretty good.

neo (udb key for scale)
neo (udb key for scale)1024×1365 308 KB

I also thought about the NVMe stuff … came to the conclusion that at least for now, the aggravation of buying and working with another point of failure (i.e. a hat) together with trying to find a non-standard case to fit it, and the additional heat, not worth the addional 50% on the IO. So I just went with my previous solution of an USB3 SSK 250G - which gives me nearly 400Mbytes/sec (read and write). This is already 2x the speed read speed and 4x the write speed of the machine I built from older PC parts a week or so ago.

One thing that worries me about cases and NVMe in general is that the CPU tops out at around 80C. Although the board can withstand 120C, NVMe devices typically quote operating temperatures “up to” 60C, so putting one “inside” an RPi case seems to be a little risky if you value your data. A failed fan might only result in a throttled CPU on the board side, but it might fry your data.

Erm, I’ve tried Arch on a Pi, maybe 2 weeks ago. Getting it going wasn’t all that easy and it doesn’t seem to be Pi optimised. For that matter, some of their security stuff seems a little stuck in time which also worried me. I know it’s very greybeard to dislike all things systemd, but I’m afraid I actually get on quite well with it.

At the end of the day PiOS “is” Debian (they’ve just tweaked it a bit) so at the moment, I’m sticking with PiOS (Debian). For the box running the forums on test I’ve just put a net install base and added INCUS from builder’s repo’s and it seems pretty happy.

Note; I had problems for quite a while with graphics and a memory leak meaning I had a Gnome crash every 2-3 days. I think this was partially a wayland issue, however the fix seems to have come from a Pi Firmware upgrade. It now seems pretty good, not had a crash for a while, this machine is showing 10 days uptime.

11

MP,

My two 5s, just arrived… :wink:

I’ve install PI-OS, if only for ease and support… but two pretty sounds reasons in my book! :slight_smile:

regards drives - I’m just using standard USB drives - the low profile ones, but 256gb…

well - that’s my weekend sorted :wink:

12

Low profile USB drives? (that’s like saying “I bought a car” :wink: )

So do you mean USB drives, or USB keys … and do you mean disk drives (HDD) or Solid State drives (SDD) ?

a quick;

sudo hdparm -tT /dev/sda

Will tell all! (well, it won’t, but it’ll give an idea)

This is my (very heavily loaded) desktop …

$ sudo hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   5642 MB in  2.00 seconds = 2822.65 MB/sec
 Timing buffered disk reads: 1086 MB in  3.00 seconds = 361.60 MB/sec

It’s the second number that’s the interesting one …

13

OK, OK… :slight_smile:

like this: Amazon.co.uk

14

Yeah, USB key … If you run a benchmark on it I’d be interested in the results, specifically write speed. Also be interested in how long it lasts, as I understand it, wear levelling on USB keys is limited. For the ones I’ve tried, the performance SD cards come out on top …

Warranty can also be interesting, worth checking what it’s covered for in terms of use-case. For SD cards for example, the “lifetime warranty” often seems only to be extended towards quoted “normal use cases”, for use in camera’s etc. Using it in a computer for example can void the warranty.

15

For a more comprehensive benchmark;

apt install fio

Then;

fio --name TEST --eta-newline=5s --filename=fio-tempfile.dat --rw=read --size=500m --io_size=10g --blocksize=1024k --ioengine=libaio --fsync=10000 --iodepth=32 --direct=1 --numjobs=1 --runtime=60 --group_reporting
...
READ: bw=368MiB/s (385MB/s)
...
fio --name TEST --eta-newline=5s --filename=fio-tempfile.dat --rw=write --size=500m --io_size=10g --blocksize=1024k --ioengine=libaio --fsync=10000 --iodepth=32 --direct=1 --numjobs=1 --runtime=60 --group_reporting
...
WRITE: bw=257MiB/s (269MB/s)
...

This should give some raw numbers that aren’t fudged by caching etc …

16

Good shout!

My 5 is still in the box at present, but I’ll report back when I have it running (or i could throw the usb into a different pc, I guess)

I have always used cheap (good branded) usbs… i accept they are not the quickest, and risk failure - but being mindful of the usage, i fund them reliable. …and cheap - all of which ensures I nail down backups! ;))

but I do like the super fast drives from your post - need to see if they do one that fits in a standard 2.5hdd slot.

have a good weekend ya’ll!

17

Ok, so this is my preferred option at present;

https://amzn.eu/d/9Ox68Cn

It’s not “that” much more expensive than what you’re using, however as far as I can see it’s sold as a “proper” SSD and it’s performance outstrips any other USB devices I’ve tried. It also comes with really good RoHS and EMC documentation, not really all that useful for end-users, but it does mean they’re putting the effort in and critical of you want to sell it on.

Performance seems to be 2-3x the Pi’s USB limits so probably still good for the Pi6 when it comes out … :wink:

18

Ok, so did anybody notice? :scream:

root@linuxforums:~$ lscpu
Architecture:            aarch64
  CPU op-mode(s):        32-bit, 64-bit
  Byte Order:            Little Endian
CPU(s):                  4
  On-line CPU(s) list:   0-3
Vendor ID:               ARM
  Model name:            Cortex-A76
    Model:               1
    Thread(s) per core:  1
    Core(s) per cluster: 4
    Socket(s):           -
    Cluster(s):          1
    Stepping:            r4p1
    CPU(s) scaling MHz:  62%
    CPU max MHz:         2400.0000
    CPU min MHz:         1500.0000
    BogoMIPS:            108.00
19

Ok, interesting … response times are slightly better when hosted in my office than when hosted in a London Data Centre …

Screenshot from 2024-04-21 04-46-31
Screenshot from 2024-04-21 04-46-311012×541 56.6 KB

Broadband seems to have come a long way …

20

Would not have noticed any change unless you told me, mind you broadband speeds are difficult to judge. After a long wait my new pc arrived this week with MX Linux installed, one of things I looked at when setting it up was the MX Repo Manager, which allows you to choose the repo you connect to for updates.

There are two MX repos in the UK one in London and one in Manchester, as I live in the Midlands about half way between the two I decided to click on the “select fastest repo” button and let that choose.

As a result I am now connected to Ede in the Netherlands !!