New Ohio Exonym Just Dropped

Tell me your game development is done outside of the United States without telling me your game development is done outside of the United States.

As an alumnus of the University of Michigan, I approve of the association of Ohio with "Calamity".

New Google Blogger Features ?!?

**Try our New Beta Features**: Create a more engaging reading experience with the help of Google

Google Search previews: Easily insert visual Google Search previews for popular people, locations, pop-culture and more directly in your blog! In Compose View, look for the ‘G’ button in the editor tool bar to get started.

That is the notice greeting me at the top of draft.blogger.com today. After years of not noticing any change in the service at all, it is now getting search previews.

Honestly I would have expected any sudden burst of activity in Google Blogger to be more distinctly AI-related, part of someone's promotion packet to sprinkle LLMs anywhere and everywhere.

Try Turning the Train Off and Back On

Today I rode the Bay Area Rapid Transit train from the peninsula up to San Francisco. Our stop at the 24th and Mission station was unusually long. The conductor announced on the speaker that they were rebooting part of the train computer.

The universal first troubleshooting step now extends to turning the train off and back on.

Training Gemma3-270m for German Q-and-A

Google recently introduced Gemma3-270M, a smaller Gemma3 model with "only" 270 million parameters instead of billions.

The most interesting aspect of this model to me is that it is explicitly intended to be able to run locally, without requiring highly specialized infrastructure — well within what is achievable outside of specialized datacenters. The potential to run the model with an air gap, isolating it from outside, would be interesting for some future stuff I'm working on.

The eventual uses would involve communication in the German language, so I decided to see about adding training to answer questions in German specifically. I referenced an existing colab notebook, which uses Gemma3-270M to predict chess moves. Chess as an application for LLMs isn't as interesting for me personally, we have better ways to use neural networks to play chess, but the training flow is the same.

We start by loading dependencies and instantiating the gemma-3-270m-it model.

%%capture
import os
if "COLAB_" not in "".join(os.environ.keys()):
    !pip install unsloth
else:
    # Do this only in Colab notebooks! Otherwise use pip install unsloth
    !pip install --no-deps bitsandbytes accelerate xformers==0.0.29.post3 peft
    !pip install --no-deps trl triton cut_cross_entropy unsloth_zoo
    !pip install sentencepiece protobuf "datasets>=3.4.1,<4.0.0" "huggingface_hub>=0.34.0" hf_transfer
    !pip install --no-deps unsloth


from unsloth import FastModel
import torch
max_seq_length = 2048
model, tokenizer = FastModel.from_pretrained(
    model_name = "unsloth/gemma-3-270m-it",
    max_seq_length = max_seq_length, # Choose any for long context!
    load_in_4bit = False,  # 4 bit quantization to reduce memory
    load_in_8bit = False, # [NEW!] A bit more accurate, uses 2x memory
    full_finetuning = False, # [NEW!] We have full finetuning now!
    # token = "hf_...", # use one if using gated models
)

We set it up to accept training data in a chat format using the Huggingface deepset/germanquad dataset, a curated set of training data from the Deutsch Wikipedia and various academic sources.

model = FastModel.get_peft_model(
    model, r = 128,
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj",],
    lora_alpha = 128, lora_dropout = 0, bias = "none",
    use_gradient_checkpointing = "unsloth",
    random_state = 3407, # Seems pretty random
    use_rslora = False, loftq_config = None,
)

from unsloth.chat_templates import get_chat_template
tokenizer = get_chat_template(tokenizer, chat_template = "gemma3")

from datasets import load_dataset
dataset = load_dataset("deepset/germanquad", split = "train[:10000]")

def convert_to_chatml(example):
    return {
        "conversations": [
            {"role": "system", "content": example["context"]},
            {"role": "user", "content": example["question"]},
            {"role": "assistant", "content": example["answers"]["text"][0]}
        ]
    }
dataset = dataset.map(convert_to_chatml)

def formatting_prompts_func(examples):
   convos = examples["conversations"]
   texts = [tokenizer.apply_chat_template(convo,tokenize = False,
       add_generation_prompt = False).removeprefix('<bos>') for convo in convos]
   return { "text" : texts, }
dataset = dataset.map(formatting_prompts_func, batched = True)

from trl import SFTTrainer, SFTConfig
trainer = SFTTrainer(
    model = model, tokenizer = tokenizer,
    train_dataset = dataset, eval_dataset = None,
    args = SFTConfig(
        dataset_text_field = "text",
        per_device_train_batch_size = 8,
        gradient_accumulation_steps = 1,
        warmup_steps = 5, num_train_epochs = 1,
        max_steps = 100, learning_rate = 5e-5,
        logging_steps = 1, optim = "adamw_8bit",
        weight_decay = 0.01, lr_scheduler_type = "linear",
        seed = 3407, output_dir="outputs",
        report_to = "none",
    ),
)

from unsloth.chat_templates import train_on_responses_only
trainer = train_on_responses_only(
    trainer,
    instruction_part = "<start_of_turn>user\n",
    response_part = "<start_of_turn>model\n",
)

We then train the model. This took about three minutes on Google Colab using a Tensor T4 system.

trainer_stats = trainer.train()

Now, the real test: can it give good answers to questions not in its training data?

messages = [
    {'role': 'system','content': 'Bielefeld'},
    {"role" : 'user', 'content' : 'Gibt es Bielefeld?'}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize = False,
    add_generation_prompt = True, # Must add for generation
).removeprefix('<bos>')

from transformers import TextStreamer
_ = model.generate(
    **tokenizer(text, return_tensors = "pt").to("cuda"),
    max_new_tokens = 125,
    temperature = 1, top_p = 0.95, top_k = 64,
    streamer = TextStreamer(tokenizer, skip_prompt = True),
)

<bos><start_of_turn>user
Gibt es Bielefeld?
<end_of_turn>

<start_of_turn>model
Ja
<end_of_turn>

Indeed yes, it can!

If that interaction doesn't make much sense: it is a German joke, alleging that the city of Bielefeld doesn't actually exist. Wikipedia has an explanation in English.

The trained model says that Bielefeld does exist. Clearly it has no sense of humor.

sync; sanc; sunc

# sync; sync; sync
#

Pros: Works.
Cons: Boring.
 

---

# cat ~/.profile
alias sanc=sync
alias sunc=sync
# 
# 
# sync; sanc; sunc

Much better.

Giant Airplanes Flying Low at 3am

Why yes, a Boeing 747 flying low directly over our house at 3am does wake us up. Who could possibly have predicted it?

Google Cloud Four Cent Bill

I'd say not to spend it all in one place but I think the maximum number of places they could possibly spend it is four, so... go ahead. Live your best life, Google, with my compliments.

YOLO Credit Union

This is a legitimate credit union in Davis, California, I just find it amusing.

:cpu: emoji reaction

I agree, Slackbot.

Kthubernetes

Inconceivable: zero hits for "Kthubernetes" ?

Unacceptable. This will not stand.

Flatulenating, Wherein We Attempt to Rectify a Dictionaric Injustice

Flatulenating: having the property of inducing flatulence.

Example: "Beans are flatulenating. I get such terrible gas every time I eat them."

 

---

 

At the time of this writing on August 10th, 2018, https://www.google.com/search?q="flatulenating" shows zero results. This blog post is an attempt to resolve this dictionaric injustice.

Bad Physics Jokes

Recently I posted a number of truly terrible physics jokes to Twitter, as one does. For your edification and bemusement, here they are:

• If you accelerate toward the red light fast enough, the blue shift turns it green again.
• Two people are walking up a frictionless hill...
  (that's it. That's the joke.)
• Whenever a neutron asks the price of anything: "For you, no charge."
• Whenever the proton is asked if they are sure: "I'm absolutely positive."
• The electron isn't invited to the party in the nucleus. The other particles find it boorish, being so constantly negative all the time.
• The Higgs Boson conveys the gravity of the situation to other particles.
• Photons make light of EVERYTHING. They can be SO inappropriate sometimes.
• You might assume that Gravitons would be extroverts attracted to large groups, but no. They're actually really, really shy.
• Despite the occasional unverified sighting, experts agree that Phlogistons are the Bigfoot of the subatomic particle world.
• Neutrinos are the tragic poets of the subatomic world. They yearn for interaction, but know that it can never be.
• As a community, Protons realized that their diet and exercise habits needed to improve.
• Other particles wish they could help the Pion, but don't know what to do. Even the smallest thing can make them fall to pieces.

Peer review terminology

peerception [peer-sept-shun] (noun) : I need to come up with some more Strengths about this person to balance out all these Areas for Improvement.

peer annum [peer-ann-uhm] (noun) : Didn't I review this person last year, and the year before that, and the year before that...

peer capita [peer cap-ee-tah] (noun) : I have how many reviews left to write?

peergatory [peer-guh-tohr-ee] (noun) : The set of reviews I might decline for lack of time.

peersuasion [peer-sway-zhun] (noun) : Do your best. They're up for promotion.

peeriodic [peer-ee-od-ik] (adjective) : Maybe I'll just copy some of what I wrote last year.

peerjury [peer-juh-ree] (noun) : This self-assessment seems somewhat... inflated.

peerritation [peer-ree-tay-shun] (noun) Unreasonable hostility felt toward the subject of the last peer review left to be written.

peersuasion [peer-sway-zhun] (noun): Do your best. They're up for promotion.

peerpetuity [peer-peh-too-it-tee] (noun): This one is taking a while to write.

peerdition [peer-dih-shun] (noun): This person really, really shouldn't have asked for my review.

peerfunctory [peer-funk-tor-ee] (adjective): Sorry, I ran out of time.

peerrihelion [peer-ih-heel-ee-un] (noun): The point at which 50% of requested peer reviews have been completed.

peerfectionist [peer-feck-shun-ist] (adjective): I spend a long time obsessing over wording.

peerfumed [peer-fume-d] (adjective): This self-assessment rather obviously glosses over failings.

peerinatal [peer-ee-nate-al] (adjective): A started but not yet completed review.

peeripeteia [peer-ih-pet-ee-ah] (noun): It was a glowing review until I remembered that outage...

peeripheral [peer-ih-fur-all] (noun): Concentrating on their 20% project.

peerquisite [peer-kwi-site] (adjective): TLs have to review their team. It's only fair.

peersecuted [peer-suh-cute-ted] (adjective): How nice. Everyone asked for my review.

peerseverance [peer-suh-veer-unce] (noun): Just keep writing, it will be done soon.

peersnickety [peer-snick-it-tee] (adjective): The self assessment could have been better.

peersona non grata [peer-so-na-non-gra-ta] (noun): Declined.

peerplexed [peer-plex'd] (adjective): What exactly did they work on, anyway?

(building on a list from a few years ago)

BangIP Option

We're almost out of IPv4 addresses, yet IPv6 deployment is still very, very slow. This is a recipe for disaster. I'm talking End of Internet predicted, film at 11 scale disaster. Something must be done. Steps must be taken.

I humbly suggest that the solution is really quite simple: re-use IPv4 addresses. I don't mean reclaiming IP addresses which have been allocated but are not actually being used. That is the kind of solution which sounds good and righteous to say but is completely unworkable in practice. Instead, I mean to simply issue the same IP address to multiple people.

"How could that possibly work?" people might ask. Go ahead, ask it... I'm glad that you asked, because I have a ready-made explanation waiting. We will leverage a solution to a similar problem which has scaled tremendously well over the last several decades: email addresses.

On early email systems like AOL, addresses had to be unique. This led to such ridiculous conventions as appending a number to the end of a popular address, like CompuGuy112673 (because really, everybody wants to have an email address like CompuGuy). The beauty of Internet email addressing is in breaking them up into a federated system, so compuguy@foo.com and compuguy@bar.com can simultaneously exist.

Therefore I propose to use this same solution for IP addressing. We will allow each Autonomous System to maintain its own IP address space. The same IP address can simultaneously exist within multiple ASNs. We are effectively adding a level of indirection to the destination in the IP header. Disambiguating the actual destination will be done via a new IP option.

Binary encoding of protocol headers is now passé. Therefore our new IP option is ASCII encoded, which also allows us to avoid the discussion about how Little Endian CPUs have won and all headers should be LE instead of BE. The content of the option is the destination IP address, followed by the '@' sign, followed by the decimal Autonomous System Number where the recipient can be reached. For example, a destination of 4.4.4.4 within the Level 3 network space would be "4.4.4.4@3356" using Level3's primary ASN of 3356.

By my reading of the IP spec the option field can be up to 40 bytes, while an IP@ASN encoding could require up to 15 bytes for the IP address, 1 byte for the @ symbol, and could handle billions of IP namespaces using only 13 more bytes to encode a 32 bit ASN. So, we should be good to go.

Updates to the DNS A record and all the application code which looks up IP addresses is left as an exercise for the reader.

For historical reasons, this new option is referred to as the BangIP option. An earlier version of this system used bang-separated IP addresses as a source routing path. That portion of the proposal has been deprecated, but we retain the name for its sentimental value.

On Octal

An article in the March 8 issue of the journal PLoS Computational Biology (as reported by Science Daily) states:

Upon pre-synaptic excitation, calcium ions entering post-synaptic neurons cause the snowflake-shaped CaMKII to transform, extending sets of 6 leg-like kinase domains above and below a central domain, the activated CaMKII resembling a double-sided insect. Each kinase domain can phosphorylate a substrate, and thus encode one bit of synaptic information. Ordered arrays of bits are termed bytes, and 6 kinase domains on one side of each CaMKII can thus phosphorylate and encode calcium-mediated synaptic inputs as 6-bit bytes.

From this we can derive one inescapable conclusion: DEC was right about octal all along.

(Thanks to Sean Hafeez for posting a link to the Science Daily article on Google+)

Refactoring Is Everywhere

The utilities used to run from poles, now they are underground. The functionality is unchanged, but the implementation is cleaner.

Multiple Inheritance

Peer Review Terminology

Many companies now include peer feedback as part of the annual review process. Each employee nominates several of their colleagues to write a review of their work from the previous year. If you are new to this process, here is some terminology which may be useful.

peerrihelion [peer-ree-heel-yun] (noun)	The point at which 50% of requested peer reviews are complete.
peersuasion [peer-swey-zhuhn] (noun)	A particularly glowing peer review.
peerjury [peer-juh-ree] (noun)	An astonishingly glowing peer review.
peerplexed [peer-plekst] (adjective)	What exactly did they work on, anyway?
peerrational [peer-rash-uh-nl] (adjective)	Why am I reviewing this person?
peergatory [peer-guh-tohr-ee] (noun)	The set of peer reviews which may have to be declined due to lack of time.
peerfectionist [peer-fek-shuh-nist] (noun)	I spent a long time obsessing over wording.
peeriodic [peer-ee-od-ik] (adjective)	Maybe I'll just copy some of what I wrote last year.
peerritation [peer-ree-tay-shun] (noun)	Unreasonable hostility felt toward the subject of the last peer review left to be written.
peersecute [peer-seh-kyoot] (verb)	How nice, everyone asked for my review.
peersona non grata [peer-soh-nah nohn grah-tah] (noun)	Nobody asked for my review?
peeregular [peer-reg-gyu-ler] (adjective)	An incomplete peer review, submitted anyway, just before the deadline.

Consistency At All Levels

Twitter now wraps all links passing through the service with the t.co link shortener, which I wondered about a little while ago. Twitter engineers sweat the details, even the HTTP response headers are concise:

$ curl --head http://t.co/WXZtRHC
HTTP/1.1 301 Moved Permanently
Date: Sun, 21 Aug 2011 12:17:43 GMT
Server: hi
Location: http://codingrelic.geekhold.com/2011/07/tweetalytics.html
Cache-Control: private,max-age=300
Expires: Sun, 21 Aug 2011 12:22:43 GMT
Connection: close
Content-Type: text/html; charset=UTF-8

Oh, hi. Um, how are you?

rel="twitter me"

Apparently Alyssa Milano used her Twitter account to verify her Google+ account.

XFN 1.1 defined a number of relationship attributes for links, one of which is rel="me"

rel="me" A link to yourself at a different URL. Exclusive of all other XFN values. Required symmetric. There is an implicit "me" relation from the contents of a directory to the directory itself.

Clearly, we need rel="me twitter" for situations like this.