My web site load continues to be very high, and it's clearly mainly crawlers. Does
Google Gemini know anything? “What is the
easiest way to identify a web crawler http request?”
Yes! The honest ones identify
themselves, but there are a whole lot of rogue bots that pose (via the User-Agent
specification) as an old browser. I can do without them:
$REQUESTER = $_SERVER ["HTTP_USER_AGENT"];
if (($REQUESTER == "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36")
|| ($REQUESTER == "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.61") )
{
http_response_code (403);
exit (0);
}
That works. But to my surprise, instead of lowering the load average, it increased
it significantly. Why? I'm still trying to get my head around it.
Worse, I am still getting so many requests that the network stack can't keep up:
Dec 14 08:03:41 lax kernel: sonewconn: pcb 0xfffff8006a459b70: Listen queue overflow: 193 already in queue awaiting acceptance (704 occurrences)
Dec 14 08:04:43 lax kernel: sonewconn: pcb 0xfffff8006a459b70: Listen queue overflow: 193 already in queue awaiting acceptance (403 occurrences)
Dec 14 08:05:44 lax kernel: sonewconn: pcb 0xfffff8006a459b70: Listen queue overflow: 193 already in queue awaiting acceptance (719 occurrences)
That's up to 12 rejects every second! Clearly I must do something about it. Others,
like Daniel O'Connor, suggest firewalling the sites for a short period of time. That
requires significantly more effort, including potentially a new firewall and identification
software like Fail2ban. Let's
wait and see if the crawlers get tired.
I'm off to Werribee tomorrow,
1½ hours' drive each way. Yes, I can drive via Geelong, but I know that road well. How
about a more scenic route via the Brisbane Ranges?
Google, find me a route:
Oh. Different routes! Why did it do that? And none of the routes look “right” to me. The
northern routes go via Sebastopol, which is quite a diversion from the way through Meredith, the one that I wanted
to take.
But then I read the fine print (web browser only; we don't want to worry the pain phone
users): “Avoids road closure on Mt Mercer-Dereel Rd”. Oh. That's right where I would want
to go. So it looks like Geelong after all.
Off to Werribee today for a
periodontic examination. How do I get there? Normally I would have gone via Meredith, Victoria and
Anakie. Meredith is 34
km away, but the road closure on the Dereel—Mount
Mercer road added a full 21 km to the route. But the route via Bannockburn is boring, so I
chose it anyway, running into no fewer than four road works on the way, none of which had an
obvious purpose. Still, it was interesting, though I had forgotten to start the GPS Logger
that Android had stopped for me despite my express instructions, so all I got was this toy display from
Google Maps:
How did it compare with Google Maps' estimates? Same distance, two minutes longer. After
deducting the 6 minutes spent waiting for road works, it's actually faster. And presumably
it would have been round 1 hour if I had been able to go via the Mount Mercer Road.
One of the nice things about Google Maps are the images. But to my surprise, things looked
nothing like I expected. I should possibly look more carefully.
My appointment was with Leela Movva,
whom I have not seen for over 3 years. And though he no longer had my records (which remained in
Geelong), he recalled details
about me. A relatively short examination, in which he didn't quite match Mario's gum depth
of 9 mm, but did find multiple depths of 8 mm, still far too much. Dignosis: periodontitis. He has a solution:
laser therapy, which seems to be new in Australia, though practiced (and
hopefully perfected) in California for over 20 years. The costs are eye-watering, over $3000, from which I
discover that my health will refund the princely sum of $739.90. I agreed that I would talk
about it to Mario, but I suppose I will have to do it.
After the periodontics, I had planned to look round Werribee for an East Asian grocery, but I
couldn't be bothered. Off instead to the Fresh Land Asian Food in
Geelong, where I went in July. I had a fairly extensive shopping list, and I wasn't able to find
everything, but to my surprise I couldn't even find tauge. In many ways the Fruit Shack is better.
I did, however, find a number of noodles I hadn't seen before:
More and more, the inability to read Chinese is becoming a problem. Today I took a couple
of photos of jars that interested me, using my Olympus E-PM2:
The first are something like pickled beans—I bought the right hand two. But what's the one
on the left? To be examined when I can get a legible translator. And I went to a lot of
trouble to confirm that it was in focus. So when I took the second photo, I didn't bother.
The sides are in focus. How did I mis that? Off-centre? With a real viewfinder, I would
have noticed.
This page contains (roughly) yesterday's and today's entries. I have
a horror of reverse chronological documents, so
all my diary entries are chronological. This page normally contains the last two days,
but if I fall behind it may contain more. You can find older entries in
the archive. Note that I often update a diary entry
a day or two after I write it.
Do you have a comment about something I have written? This is a diary, not a
“blog”, and there is
deliberately no provision for directly adding comments. It's also not a vehicle
for third-party content. But I welcome feedback and try to
reply to all messages I receive. See the diary overview for more details. If you do
send me a message relating to something I have written, please indicate
whether you'd prefer me not to mention your name. Otherwise
I'll assume that it's OK to do so.
