here's a small selection in various price ranges

Also: Literally anything that can run GNU/Linux (or BSD) and has 2 or more ethernet ports, in combination with a switch one already has.
Depending on what junk you have laying about (I'm currently running a PCEngines APU2 as a router, but I used an old Pentium III SFF desktop for many years prior), that could mean price ~ zero with features and flexibility superior to an off the shelf router.

More generally, details on the connection probably matter here. Pretty much any fibre connection will require a router of some kind, and some require VLAN or PPPoE support on the router as well.
The sort of works for one machine bit inclines me to think it's straight IPv4 DHCP in this case, though it may also be using VLANs for traffic shaping.

Suitable devices aplenty, here's a small selection in various price ranges:

Ubiquiti EdgeRouter X

MikroTik hEX S router

Ubiquiti EdgeRouter 6P

Various FritzBox router models, including those for direct Fibre connection.

Some models have WiFi, but you can switch that off, as I do too.

We have fiber internet now, literally just a wall ethernet jack, no modem needed, but having a problem with switches/router.

We don't want or need wi-fi, we are connecting all directly with cat5e and cat6 cablesto the switch. Our service is 300 mbps, but old laptops only have a 10/100 NIC, but no biggie even 100 mbps would be awesome.

So I have an old generic 5 port switch that works so that's what I hooked up first (direct to wall jack). It works except for some reason it throttles everything down to 25-30 mbps, it's a managed switch but lost the password to login to it a long time ago (it's an almost 20 year old switch).

Tried an older Belkin wi-fi router that has 4 ethernet outputs, outputs all worked but router would not connect to internet, I have the password to it but no amount of tinkering would let it connect.

Bought a TPlink gigabit unmanaged 5 port switch, but it will only allow one output connection when it's plugged directly into wall, apparently something to do with IP addressing, TPlink says something to the effect that it's topology won't work being plugged in directly, it needs a router. (grrrrr, aggravating).

Here's the weird thing, occasionally I can get 90-95 mbps on one machine only, by plugging the gigabit switch into the wall, plugging it's output into the older 10/100 switches input and start by running both laptops off of the 10/100, then while running, unplug one laptop and plugging it into the gigabit switch.  Doesn't work every time, and if I mess with any of it by swapping connections here and there it will stop working and i'll have to plug everything back in to the 10/100 switch. Very strange.

So I just wanted to know, does anybody know of a 5-port switch (or some kind of doodad) that CAN be plugged directly into wall jack and support the 4 outputs and give at least the 100mbps my machines can support?

Thanks for any help, been messing with this all day and have a heckuva headache now, I am lousy at this networking stuff.