Daily Google Drive backup using rclone

I still make regular use of Google's Docs, Sheets, and so on but make regular backups to a local drive which is itself backed up off-site. I've settled on the following script, run from cron in the middle of the night. It sends email on failure, and reports success on Mondays just so I'll see that it is working. It is saved as /usr/local/bin/gdrive_sync.sh.

root@pve:~# cat /usr/local/bin/gdrive_sync.sh
#!/bin/sh

tmpfile=$(mktemp)
/usr/bin/rclone sync --fast-list --transfers=32 --create-empty-src-dirs \
	--exclude-if-present=exclude-from-rclone-sync \
	--drive-acknowledge-abuse \
	denny-at-example-com-google-drive-backup: \
	/pool1/GDrive/denny@example.com > ${tmpfile} 2>&1

if [ $? -ne 0 ]; then
	echo rclone Failed
	rm -f ${tmpfile}
	exit 1
fi

extra=$(grep -v -e "Duplicate object found in source" -e "Duplicate directory found in source" ${tmpfile})
rm -f ${tmpfile}

if [ "${extra}" ]; then
	echo rclone errors found
	echo ${extra}
	exit 1
fi

dow=$(date +%w)
if [ ${dow} -eq 1 ]; then
	# Report success once per week
	echo Successfully backed up denny@example.com Google Drive
fi

exit 0

Vaccines Working Less Well

Decades ago as infants I received several doses of the polio vaccine, one of a panel of childhood vaccinations. Mass vaccination had already had a huge impact on the disease by that point, infections and serious consequences had already dropped substantially.

 

My Favorite Shirt

However there was no Varicella vaccine back then, the virus which causes chicken pox. We now know that Varicella is also responsible for the even more serious Shingles later in life, as the long-dormant virus can become active again.

I remember chicken pox. The thing I remember most about suffering through chicken pox isn't the itching, though that was bad. It isn't the blisters, it isn't the fever. The thing I remember most is that I had been allowed to wear my favorite shirt as a way to comfort me, and that shirt disappeared after I recovered. My parents said they didn't know what happened to it, it had been lost. I was upset about it, and remember the feeling of loss to this day.

It was decades later, after my parents had both passed away, that I realized: they burned the shirt. Chicken pox is so infectious that parents would often dispose of the bedding and clothing afterwards.

My children have little context for this. They never developed chicken pox, and don't know anyone who has. They've never even heard of anyone who developed chicken pox. The infection rate has dropped by 97% since the addition of Varicella to the childhood vaccine schedule in 1995.

In fact, until Covid-19, they didn't have to give much thought to disease at all.

 

Vaccines work

We're seeing the same rapid reduction in the many cancers which are triggered by the HPV virus. Rates of these cancers are dropping due to the introduction of widespread HPV vaccination a number of years ago.

Vaccines work. They work well at an individual level, but they work even better when a disease cannot gain any hold within a population — the much-discussed herd immunity. In the US we are rapidly losing this herd immunity as antivax sentiment gains strength and more parents decline to vaccinate their children.

Vaccines work but we're heading into a world where, for the first time, vaccines won't work as well as they used to.

 

Booster shots

Several weeks ago I had blood titers run to gauge antibody response from the vaccines of my childhood. This week I got booster shots for Measles/Mumps/Rubella and Polio. Because vaccines still work, but they've not going to work as well as they used to. This is the sad reality we have to prepare for.

Google Photos Takeout failure

Two months ago I exported all photos from Google Photos to import into immich, and scheduled followon Takeout runs every two months. I had naively assumed the subsequent exports would be incremental changes from the first one, but they are instead complete exports again. Oh well.

More significantly though, they seem to require re-entering my login every 20 minutes or so. When I click on a download link if I haven't logged in within the last few minutes I am asked to login again, but also downloads which do not complete within 20 minutes seem to stop. Clicking "resume" in the Chrome download window doesn't succeed.

 

Once this has happened enough times, Takeout disallows trying to download again.

 

ZIP archives #7 and #8 are now inaccessible to me.

I don't recall needing to login so often during the last export, two months ago. Though the downloads did sometimes fail, I also don't recall having the resume download fail because of the need to login again. It probably did not change in the intervening two months, but somehow I'm running into it this time.

Google Takeout really is a wonderful thing, but I'm going to have to do it again and — I guess — be especially careful about when to download the links. Do I have to be ready to click Resume immediately? That seems primitive.

LLMs for spoken language practice

One use-case for large language models that I'm especially eager to see evolve is in assisting humans in learning languages. Once one has achieved roughly A2 in a new language, progress becomes much more rapid by holding spoken conversations in the new language. It is something which an LLM should be reasonably good at:

• Large corpus of idiomatic written and spoken language available on general topics.
• Generate prompts for a few thousand hypothetical situations to navigate in the new language, as part of the curriculum.
• Doesn't require the LLM to be an expert in a factual area. Just converse.

One could offer 1:1 instruction with the human student paired with an LLM instance. The prompt could gradually adjust the sophistication of the language as the student progresses. The LLM could evaluate the human student's pronunciation and word choice, and offer tips for how to be more idiomatic.

The Running of the Proxmoxes

I update Proxmox every 1-2 years.

Today was the day.

Success!

Renewing US Passports early

If a US Passport is expiring in one year or less, there is now a very nice online portal to request a renewal. If the passport has more than a year left but one wants to renew it early, paper forms must be sent and the old passport mailed in.

We wanted to renew but our passports had a bit more than one year left, so we filled out the DS-82 forms and sent them with the old passports.

• We sent the mailer via certified mail in November, 2024.
• The new passport books were delivered to our home 17 days later.
• The new passport cards (first time!) were delivered to our home 1 day after that.

So passport renewals take less than three weeks right now, even without paying for expedited serice nor express mail shipment. The old passport books were returned a bit later, 22 days after we sent them in for renewal. We appreciate this, they had passport stamps which hold memories for us.

Installing Signal is Simple

In researching Signal Messenger I was left with the impression that it was difficult to use and tryig to get the whole family to use it would be painful. Having now finished getting it set up on everyone's phone I have to say: it was not difficult at all.

We each established a 1:1 channel with each other, and a group chat for the Family. I and one of the teenagers are admins of the Group. We've also used it for a 1:1 video call between an Android phone and an iPhone.

So far, so good.

Caltrain Electrification

Caltrain is a large publicly owned train service in California. For the past few years Caltrain has been installing electrical lines above the tracks running up the San Francisco peninsula. Train electrification was proposed thirty years ago, began work seven years ago and completed in September 2024 with the retirement of diesel and replacement by electric engines.

The electric train engines have considerably better acceleration, important given the number of stops along the peninsula, and can complete the journey from San Jose to San Francisco in 30% less time than the diesel engines could. This allows trains to be scheduled more frequently.

 

Induced Demand

Studies have previously shown that service frequency has a significant impact on ridership, and indeed, Caltrain ridership is up 54%. On weekends it is up even more, by more than double.

We talk about the Braess paradox, where adding a lane to a busy roadway will further increase traffic and result in more congestion not less. Building massive highways doesn't resolve traffic congestion for car travel.

Yet increased ridership does not trigger the Braess paradox in train networks. Inherent in the paradox is that moving entities choose their own route, and will optimize for their individual outcome even at the expense of the whole. The trains are not making individual optimization choices, they just run faster.

 

Electrified Trains are Better Neighbors

As a family we enjoy a restaurant right next to the San Carlos Caltrain station, one with outdoor seating. The most recent trip there was a revelation: the electric train is quiet while getting underway, and with no diesel fumes. Electric trains fit into the cityscape more cleanly, quietly whisking passengers on their way.

 

Tailscale certificates with NextCloud

I run a self-hosted NextCloud instance within the home, and use Tailscale to access it while out and about. This entailed editing /var/www/nextcloud/config/config.php to add trusted_domains:

'trusted_domains' =>
array (
  0 => 'localhost',
  1 => 'nextcloud.tails-scales.ts.net',
),

As using the default self-signed certificate is annoying, I installed a Tailscale certificate instead. A script run from crontab each week automatically renews the certificate:

#/bin/bash

out=$(tailscale cert --cert-file /etc/ssl/certs/tailscale.crt \
                     --key-file /etc/ssl/private/tailscale.key \
                     nextcloud.tails-scales.ts.net)

if [ $? -ne 0 ]; then
  echo tailscale cert failed
  exit 1
fi

# No new certificate needed, just quietly exit
if echo ${out} | grep -q unchanged ; then
  exit 0
fi

echo tailscale cert updated, reloading apache
systemctl reload apache2

Backup Google Photos to immich

I have over a century's worth of pictures in Google Photos. I am not immortal or anything, I just inherited a trove of family photos and digitized them all. The oldest photos are from the late 19th century.

I honestly do very much respect Google, the company and the services, but one concern about Google is the horror stories when their systems decide a user should be suspended. I want to minimize irreplaceable losses were I to lose access to my data stored there. The first steps of this have been:

1. Google Takeout of files in my Drive, to store it in a self-hosted NextCloud.
2. Google Takeout of my Photos, to store in a self-hosted immich.

This post is about the second item, getting photos into immich.

 

immich

I went with immich primarily because of its good commonality with Google Photos — I've mostly liked Google's product, just looking for more control of the ownership of my data. Immich presents a chronological view of photos, and allows organization into albums. It provides facial recognition which has honestly been top notch so far.

I host immich on a Proxmox machine at my home, and use Tailscale to get access to it from anywhere. The primary immich developers prefer to package it as a set of docker containers, which Proxmox doesn't natively support. Instead I created a Debian 12 LXC for immich and installed docker, using Docker Compose to get a working system.

 

Google Takeout for Photos

This was actually the most time-consuming step in the process: I used Google Takeout to export my photos collection and schedule five more Takeouts every two months, spanning a year in total. The first export took a few hours to prepare, and then an entire day to download. It comprised fifteen ZIP files in all, the first fourteen being 50 Gigabytes in size and the last being the rest.

The ZIP files are so large because they are full of duplicates. The entire photo library is included, and every album. Every time a photo appears in an album, an entire additional copy of it is added to the ZIP file. For over a decade I've zealously made albums for every trip to the Zoo, every school event, every trip, and the duplication of a substantial percentage of the photo library adds to the size of the download. Don't worry though, the process of importing the collection into immich will deduplicate them as well.

Being Google, the Takeout also contains JSON files for any metadata about photos it has. For example if you manually add a location Photos does not add that location to the JPEG EXIF data, it provides it in the JSON file instead.

Once downloaded, the ZIP files can be left as-is — the next step can process the ZIP files directly. You don't need to unpack them.

 

immich-go

Immich-go is an alternative CLI for immich written by Jean François Cassan. One of the things it can do is import Google Takeout ZIP files. It makes a first pass collecting images and their associated JSON. Given an immich API key it then uploads the deduplicated images, creating all Albums contained within the Takeout.

After about 12 hours running, immich0go had uploaded about 60,000 photos to immich.

 

immich facial recognition

Over the next several days, immich gradually recognized faces in photos and began associating them together. This, unfortunately but understandably, is not something exported from Google Takeout. The carefully curated set of identified people from Photos is left behind at Google, instead I'll need to recontruct it afresh within immich.

On the bright side, immich thus far appears to be very accurate in its groupings, even moreso than Google's implementation. It doesn't conflate different individuals together quite so often.

 

Next steps

I've been happy with the results thus far. Google Takeout's first scheduled incremental export a the two month mark is coming up in a couple weeks. I'll export another, presumably much smaller, Takeout and see how well immich-go handles it.

California Driver's License reciprocity with Germany current status

A number of US states and territories have full or partial reciprocal driver’s license agreements with Germany, allowing a US license to be exchanged for German and vice-versa. Among these are:

Alabama	Iowa	New Mexico	Utah
Arizona	Kansas	Ohio	Virginia
Arkansas	Kentucky	Oklahoma	West Virginia
Colorado	Louisiana	Pennsylvania	Washington
Delaware	Maryland	South Carolina	Wisconsin
Idaho	Massachusetts	South Dakota	Wyoming
Illinois	Michigan	Texas	Puerto Rico

California, notably, is not on the list. I believe the current status is as follows.

1. California Vehicle Code §12804.9 does not allow the DMV to negotiate reciprocity agreements with foreign governments, only with US territories.
2. There have been several attempts to update the Vehicle Code to allow this, most recently CA AB 639 in the 2023-2024 legislative session — which was passed out of committee but ultimately not voted on in the House and died at the end of the 2024 legislative session.
3. In the prior legislative session, AB-723 in 2021-2022 made it through several committees and the first few steps of a vote of the House, but was also not acted on before the end of the term.
4. AB-629 of the 2019-2020 session passed in one committee but did not proceed further.
5. The earliest such bill I can find came in the 2017-2018 session, and appears to also be the one which made it the furthest. SB-1360 passed the California Senate and through several committees in the House, but was not brought to a vote in the full House before the end of the legislative session.

 

The Experience in Washington State

The state of Washington passed a law allowing negotiation of reciprocal driver's licenses in 2005, in RCW 46.20.125. The full text of the change is considerably simpler than the proposed changes in California law, which additionally requires a consular letter and driving history.

Washington state and Germany signed a Memorandum of Understanding in 2005 relatively soon after passage of the enabling legislation. It was renewed in 2019, which notes the agreement as having been in place for 14 years.

 

Next Steps in California

The sponsor of the most recent Assembly Bill 639 was Representative Evan Low, who will not be returning to the state congress next year. I wish him well. I am writing to several other state representatives to encourage an effort in the next legislative session.

I believe framing this as a positive approach to mutually beneficial immigration would be productive.

There is recent momentum within the United States in this area. The American Association of Motor Vehicle Administrators publishes a guide of best practices for foreign driver’s license reciprocity, last updated in 2021 after its initial publication in 2009. It includes the treaty and legislative background for such agreements, and samples of a Standard Operating Practice and a Memorandum of Understanding. It also includes survey data of the number of agreements signed each year, which shows consistent interest over time.

If you live in California or are registered to vote in California from abroad and interested in advocating for this change in the California Vehicle Code, please get in touch: denton.gentry@gmail.com. I'm writing to representatives of how it might be carried forward.

Charles Schwab brokerage debit card

When traveling overseas a debit card from Charles Schwab is quite useful:

• no fees for use of overseas ATMs
• refunds fees charged by the owner of the ATM
• withdraw in dollars and let Schwab do the currency exchange at a favorable rate

If you apply for a Schwab debit card, by default it will be tied to a checking account with Charles Schwab Bank. If you don't have a checking account Schwab will ask permission to set one up. This is fine for many people, but if you might at some point move overseas it is a problem. Schwab offers International Brokerage accounts for US citizens living overseas, but banking regulations mean that Charles Schwab Bank cannot do so and will close a checking account if they determine that the holder resides outside of the US.

However, one can also get a Schwab debit card with the same terms and conditions but attached directly to the brokerage account and not through Charles Schwab Bank at all. A debit card attached to the brokerage account will still work if it later becomes a Schwab International Brokerage account. I had to call Schwab to make this happen, the web form didn't provide an obvious way to do so.

The card looks similar but is issued by Charles Schwab Corporation not Charles Schwab Bank, and says "Schwab One brokerage" on it.

Emergency vs Loadshifting Batteries

Our house has 8 kilowatts of solar generation and 27 kilowatt-hours of battery storage, and is on a time-of-day electricity plan where power is cheaper in the early part of the day and more expensive from 3pm until midnight. We use 70% of the battery capacity to power the house during the peak hours each day, reserving 30% in case of outage.

This does work, but one battery is filling two roles with requirements at least somewhat in conflict.

• For load shifting we would prefer to use more of the battery to reduce our energy bill.
• If there is a power outage, we'd prefer to have more of the battery held in reserve than 30%.

Yet the batteries wired in to the house are not the only substantial energy storage on site. We also have an electric vehicle, with as much battery capacity as the house. A more modern EV than ours would have even more battery capacity.

 

Vehicle to Grid

That the substantial battery power in EVs could be useful in grid emergencies has long been recognized. Vehicle-to-Grid (V2G) is an idea to make the charging port of the EV bidirectional: charging the EV from the grid as normal, but also able to supply to the grid by draining the EV. In very large numbers, EVs could provide enough power to stabilize grid operation while still having enough power to be used for transportation.

V2G has been slow to catch on, owing mainly to the number of parties involved. Because it entails connecting a new generation source to the grid, it must follow similar processes and permits as a rooftop solar installation. The timeline of a solar install is long: it can be months with roofers / installers / electricians in several waves, and having time enough to apply for the needed permits and interconnection agreement.

Yet when people buy an EV they need a way to charge it almost immediately. Waiting to install an EV charger isn't realistic, so it will be installed to only draw from the grid not supply back to the grid. Designing V2G capabilities into an EV charger in hopes the homeowner will followup with additional permitting? Most consumers wouldn't ever do so.

 

Vehicle to Grid Vehicle to LOAD

Vehicle-to-Load (V2L) is an idea with fewer stakeholders involved: allow the vehicle to supply power to locally connected loads, most commonly by having electrical outlets and an inverter built into the car. This is useful in many situations, like an EV at a job site powering tools or while camping or picnicking.

V2L also allows the EV's stored power to be used at home during emergencies, albeit somewhat awkwardly. It doesn't power the whole house, it can power appliances which are unplugged from the house and plugged into an extension cord from the vehicle. Keeping cell phones charged, running medical equipment, or powering freezers full of stored food is quite feasible. Running central air conditioning is not.

This concept works well for emergencies in that it can power essential needs from a very large battery, especially because the battery is mobile and could go somewhere to charge itself if needed. Natural disasters in the last several years have demonstrated this, notably Hurricane Helene in the US Southeast just last week. EVs are helping supply power in damaged areas.

California, my home state, has a large enough market for automobiles that it has often been able to influence the auto industry throughout the US such as via fuel efficiency standards. A law being considered would allow the state to mandate Vehicle-to-Load capabilties for vehicles sold as of some future date.

 

Transfer switch

We'd very much want our next EV to have this capability. Our use of the batteries built into the home would change to entirely load shifting to further reduce our demand on the grid during peak hours. During a power outage of appreciable length, we'd rely on the much larger batteries in the vehicle to keep food frozen and cell phones charged.

For this to work we'd need to plug appliances into an extension cord from the vehicle, and some of them are hard-wired into the house. The traditional way to handle this is a transfer switch, intended for a generator. This is expensive, and difficult to retrofit. There needs to be an easier way.

Powerwalls and Time Based Controls

We are on a Time-of-Day electricity plan, the PG&E EV2 plan. Our cost for electricity is:

		Summer	Winter
Off-Peak	12am - 3pm	$0.31/kWh	$0.31/kWh
Mid-Peak	3pm - 4pm	$0.51/kWh	$0.48/kWh
Peak	4pm - 9pm	$0.62/kWh	$0.49/kWh
Mid-Peak	9pm - 12am	$0.51/kWh	$0.48/kWh

We are incentivized to keep the power-hungry activities like charging the electric vehicle to the daylight hours where solar is plentiful or overnight where demand is low. Unfortunately however, there are several confounding factors to our energy use which make things more complicated.

1. The house is on a hill, the crest of which blocks direct sunlight for longer and longer periods each winter day.
2. We have two Powerwalls, able to store 27 kWh to supply the house at other times.
3. BUT, when we installed the powerwalls in 2019 we claimed the Residential Clean Energy tax credit which requires the batteries be charged using only solar power for five years.

Combining all of these things, we ended up with an unfortunate confluence in the winter months when the hill allows only a few hours of direct sunlight: there is not enough excess solar to charge the batteries while also powering the house.

We would head into the higher priced times of day with little ability to time shift stored solar production. The batteries were never able to charge. Our power bills rose substantially, calling into question why we paid for this stuff in the first place.

The Tesla app has a "Time Based Control" mode, where it takes the time of day and rate plan into account. However its main focus is in exporting solar production during high value hours by running the home from battery. Lacking sufficient production to charge the battery, this resulted in poor outcomes with Time Based Control. I didn't look at it again for the next few years.

 

Hacking Around It

Instead we've come up with techniques to get things working acceptably:

1. On winter mornings set the Powerwall to reserve 100% of its capacity for power outages. All solar production during the day charges the batteries, trying to reach 100%. The home's needs are met from the grid during off-peak hours.
2. When peak hours start, change the Powerwall to 30%. It then discharges to power the home, allowing the evening load to be partially met using stored solar power.
3. Do this every day. Change the Powerwall setting in morning and afternoon, every day, all winter.

I of course wrote software to automate this, but Tesla has only recently decided to offer an actual API to control Powerwalls. For the first few years I was instead using authentication mechanisms and APIs which a community on GitHub would reverse engineer, and which Tesla kept deliberately breaking. I had to watch for errors from my software or, maddeningly, when it would run without error but Tesla ignored its commands.

 

Changing the Game

At the beginning of this month, our five years was finally reached. I set the Powerwall to be able to charge itself using grid power. This didn't immediately change much behavior, until I toggled it to Time Based Control again.

Now, suddenly, things are much improved. At 3pm every day the Powerwall begins supplying the house energy demand, and all remaining solar production is sent to the grid. At midnight when rates drop, the Powerwall charges itself to 100% using grid power, to be ready for the next day.

This is already better than the system I had cobbled together:

• There were many days when the total solar production could not fully charge the batteries. Now, no matter what, the batteries are 100% charged every day.
• We are on the Net Energy Metering plan from 2019. Sending solar to the grid offsets our use at other times, so long as the house can be powered from battery.

 

Futures

I do want to change the current behavior in one way: instead of charging from the grid overnight, I'd prefer the Powerwall try to charge from excess solar. In summer it will usually be able to do so, and in winter it can try and then start charging itself mid-morning from the grid if it isn't going to make it to full. I'm looking into the Fleet API which Tesla published this year for what might be possible.

However, fundamentally, this stuff needs to be easier for the homeowner. I've been writing custom software and manually intervening for years, just to get a decent result out of it. I should not need to do that, after having paid so much for the system install.

One small provision in the Inflation Reduction Act was to remove the five year solar charging mandate for batteries installed after its passage in 2023; the goal is to incentivize more batteries on the grid. With freedom to charge the battery, it should be able to figure out how and when to charge itself. The behavior of the system over time should inform future operation, deciding when to charge from the grid and when to trust that solar power will provide. Next-day solar forecasts can inform this decisionmaking.

Surplus Energy Response

The phrase "too cheap to meter" entered the energy discourse in the 20th century, referring to the potential of nuclear power. Though originally coined to refer to fusion power, the label instead became associated with all nuclear energy. Seventy years later fusion is not here yet, and fission power has been a solid source of baseload power but could never be described as cheap.

The idea of energy too cheap to meter is a compelling one, we just had to wait for an entirely different technology to deliver it: solar photovoltaic. Panels installed on residences and commercial buildings are typically installed "behind the meter," where it directly supplies the energy demand of the building. The proof that solar power is too cheap to meter is that is not, in fact, metered.

Solar deployment has grown incredibly rapidly, faster than the distribution grid would be ready to accept it all. Deployment behind the meter has been essential because the grid in front of the meter hasn't been able to deploy new capacity so rapidly, and solar deployment continues to accelerate. A post by Ben James argues that solar energy can be deployed so inexpensively that using it completely off the grid, for economic activities which can be economical with free energy so long as it can handle being run intermittently only when the sun is shining, is compelling.

• hydrogen production, via electrolysis of water
• fertilizer production, producing ammonia via air capture and energy
• kerosene production, also via air capture
• ... and other chemical processes made possible by prolific free energy

 

Surplus Energy Response

The electric grid has a notion of Demand Response, when there is heavy demand which stresses available generation — for example, by air conditioning on a hot afternoon. We have reached the point where we also have the opposite situation: we need a surplus response. Many builings now produce substantial excess behind-the-meter power during the day, so much so that the grid cannot absorb it all. We need our buildings to become smarter about putting the excess energy to useful work:

• pre-heat or pre-cool HVAC, somewhat overshooting the temperature setpoint while energy is free
• store hotter water, with a smart water system to mix scalding with cold to get the desired water temperature
• charge electric vehicles for free, with knowledge of when the vehicle is likely to be needed
• charge up batteries in appliances throughout the building, allowing high peak load appliances to be installed in buildings not originally built for them

Yet we can take it even further. Limitless free energy, albeit at limited time ranges within a day, allows us to make choices we would never have otherwise considered.

• heat a pool or hot-tub to be ready for impromptu human use at any time during summer months
• run heat pumps in a sunroom, open to fresh air yet maintained at a comfortable temperature
• indoor urban hydroponics, pre-engineered gardens which are never too hot nor cold and provide generous fresh produce

In building enthusiasm for the energy transition, providing services which seem impossibly luxurious yet are provided entirely by surplus energy would be a compelling outcome.