So no, no billion dollar company can make their own training data

This statement brought along with it the terrifying thought that there’s a dystopian alternative timeline where companies do make their own training data, by commissioning untold numbers of scientists, engineers, artists, researchers, and other specialties to undertake work that no one else has. But rather than trying to further the sum of human knowledge, or even directly commercializing the fruits of that research, that it’s all just fodder to throw into the LLM training set. A world where knowledge is not only gatekept like Elsevier but it isn’t even accessible by humans: only the LLM will get to read it and digest it for human consumption.

Written by humans, read by AI, spoonfed to humans. My god, what an awful world that would be.

A few factors:

• Human population centers historically were built by natural waterways and/or by the sea, to enable access to trade, seafood, and obviously, water for drinking and agriculture
• When the fastest mode of land transport is a horse (ie no railways or automobiles), the long-distance roads between nations which existed up to the 1700s were generally unimproved and dangerous, both from the risk of breakdown but also highway robbery. Short-distance roads made for excellent invasion routes for an army, and so those tended to fall under control of the same nation.
• Water transport was (and still is) capable of moving large quantities of tonnage, and so was the predominant form of trade, only seeing competition when land transport improved and air transport was introduced.

So going back centuries when all the “local” roads are still within the same country (due to conquest), and all the long-distance roads were treacherous, slow, and usually uncomfortable (ie dysentery on the Oregon Trail), the most obvious way to get to another country would have been to get a ride on a trading ship. An island nation would certainly regard all other countries as being “overseas”, but so would an insular nation hemmed in by mountains but sitting directly on the sea. When land transport is limited, sea routes are the next best. And whereas roads only connect places situated along the route, the sea (and the sky) allow point-to-point trading, exposing faraway countries to each other when their ships arrive at the port.

TL;DR: for most of human history, other countries were most reasonably reached by sea. Hence “overseas”.

Does anyone want to even guess what time this clock is supposed to be indicating?

Truly, it could be anything that unsettles the market. A bubble popping is essentially a cascading failure, where the dominos fall, when the house of cards collapses, when fear turns into panic, even when everyone is of sound mind.

The Great Depression is said to have started because of a colossally bad “short squeeze”, where investors tried to corner the market on copper futures, I think. That caused some investment firms to go broke, which then meant trust overall was shaken. And then things spiraled out of control thereafter, irrespective of whether the underlying industries were impacted or not.

So too did the Great Financial Crisis in 2008, where the USA housing market collapsed, and the extra leverage that mortgagees had against their home value worked against them, plunging both individuals and mortgage companies into financial ruin. In that situation, the fact that some people lost their homes, coupled with them losing their jobs due to receding market, was an unvirtuous cycle that fed itself.

I can’t speculate as to what will pop the current bubble, but more likely than not, it will be as equally messy as bubbles of yore. But much like the Big One – which here in California refers to another devastating earthquake to come – it’s not a question of if but when.

Until it (and the AI bubble popping) happens though, it is not within my power to do much about it, and so I’ll spend my time preparing. That doesn’t mean I’m off to move my retirement funds into S&P500 ex-AI though, since even the Dot Com bubble produced gains before it went belly up. I must reiterate that no one knows when the bubble will pop, so getting on or getting off now is a financial risk.

Preparation means to build resilience, to decouple my home from my job, to keep my family and community safe even when the shaking starts. For some, this means stocking food and water. For others, it means building mutual aid networks. And for some still, it means downsizing and making their lives more financially sustainable, before the choice is made for them.

This is a rollercoaster and we’re all strapped in, whether we like it or not.

I’ll take a stab at the question, but we will need to dive into a small amount of computer engineering to explain. To start, I am going to assume an x86_64 platform, because while ARM64 and other platforms do support hardware-virtualization, x86_64 is the most popular and most relevant since its introduction at the beginning of the 2000s. My next assumption is that we are using non-ancient hardware, for reasons that will become clear.

As a base concept, a virtual machine means that we have a guest OS that runs subordinate to a host OS on the same piece of hardware. The host OS essentially treats the guest OS as though it is just another userspace process, and gives the guest some time on the CPU, however the host sees fit. The guest OS, meanwhile, is itself a full-blown OS that manages its own userspace processes, divying out whatever CPU time and memory that it can get from the host OS, and this is essentially identical to how it would behave if the guest OS were running on hardware.

The most rudimentary form of virtual machine isolation was achieved back in the 1960s, with software-based virtual machines. This meant that the host emulated every single instruction that the guest OS would issue, recreating every side-effect and memory access that the guest wanted. In this way, the guest OS could run without change, and could even have been written in an entirely different CPU architecture. The IBM System/360 family of mainframes could do this, as a way of ensuring business customers that their old software could still run on new hardware.

The drawbacks are that the performance is generally less-than-stellar, but in an era that valued program correctness, this worked rather well. Also, the idea would carry into higher level languages, most notably the Java Virtual Machine (JVM). The Java language generally compiles down to bytecode suitable to run on the JVM (which doesn’t really exist), and then real machines would essentially run a JVM emulator to actually call the program. In this way, Java is a high-level language that can run anywhere, if provided a JVM implementation.

An advancement from software virtualization is hardware-assisted virtualization, where some amount of the emulation task is offloaded to the machine itself. This is most relevant when virtualizing the same CPU architecture, such as an x86_64 guest on an x86_64 host. The idea is that lots of instructions have no side-effects that affect the host, and so can be run natively on the CPU, then return control back to the host when reaching an instruction that has side-effects. For example, the basic arithmetic operation of adding two registers imposes no risks to the stability of the machine.

To do hardware-assisted virtualizaton requires that the hardware can intercept (or traps) such instructions as they appear, since the nature of branch statements or conditionals means that we can’t detect in-advance whether the guest OS will issue those instructions or not. The CPU will merrily execute all the “safe” instructions within the scope of the guest, but the moment that it sees an “unsafe” instruction, it must stop and kick back control to the host OS, which can then deal with that instruction in the original, emulated fashion.

The benefit is that the guest OS remains unmodified (yay for program correctness!) while getting a substantial speed boost compared to emulation. The drawback is that we need the hardware to help us. Fortunately, Intel and AMD rose to the challenge once x86-on-x86 software virtualization started to show its worth after the early 2000s, when VMWare et al demonstrated that the concept was feasible on x86. Intel VT-x and AMD-V are the hardware helpers, introducing a new set of instructions that the host can issue, which will cause the CPU to start executing guest OS instructions until trapping and returning control back to the host.

I will pause to note why same-on-same CPU architecture virtualization is even desirable, since compared to the emulation-oriented history, this might not seem immediately useful. Essentially, software-based virtualization achieved two goals, the latter which would become extremely relevant only decades later: 1) allow running a nested “machine”, and 2) isolate the nested machine from the parent machine. When emulation was a given, then isolation was practically assured. But for same-on-same virtualization, the benefit of isolation is all that remains. And that proved commercially viable when silicon hit a roadblock at ~4 GHz, and we were unable to make practical single-core CPUs go any faster.

That meant that growing compute would come in the form of multiple cores per CPU chip, and this overlapped with a problem in the server market where having a separate server for your web server, and database server, and proxy server, all of these cost money. But seeing as new CPUs have multiple cores, it would save a bunch of money to consolidate these disparate servers into the same physical machine, so long as they could be assured that they were still logically running independently. That is to say, if only they were isolated.

Lo and behold, Intel VT-X and AMD-V were introduced just as core counts were scaling up in the 2010s. And this worked decently well, since hardware-assisted virtualization was a fair order of magnitude faster than trying to emulate x86, which we could have done but it was just too slow for commercialization.

Some problems quickly emerged, due to the limitations of the hardware assistance. The first has to do with how the guest OS expects to operate, and the second to do with how memory in general is accessed in a performant manner. The fix for these problems involves more hardware assistance features, but also relaxing the requirement that the guest OS remain unchanged. When the guest OS is modified to be better virtualized, this is known as paravirtualization.

All modern multi-tasking OS with non-trivial amounts of memory (which would include all guest OS’s that we care about) does not organize its accessible memory as though it were a "flat’ plane of memory. Rather, memory is typically “paged” – meaning that it’s divvied out in pre-ordained chunks, such as 4096 Bytes – and frequently also makes use of “virtual memory”. Unfortunately, this is a clash in nomenclature, since “virtual memory” long predates virtualization. But understand that “virtual memory” means that userspace programs won’t see physical addresses for its pointers, but rather a fictional address which is cleverly mapped back to physical addresses.

When combining virtual memory with pages, the OS is able to give userspace programs the appearance of near-unlimited, contiguous memory, even though the physical memory behind those virtual addresses are scattered all over the place. This is a defining feature of an OS: to organize and present memory sensibly.

The problem for virtualization is that if the host OS is already doing virtual+paged memory management, then it forces the guest OS to live within the host’s virtual+paged environment, all while the guest OS also wants to do its own virtual+paged memory management to service its own processes. While the host OS can rely upon the physical MMU to efficiently implement virtual+paged memory management, the guest OS cannot. And so the guest OS is always slowed down by the host having to emulate this job.

The second issue relates to caching, and how a CPU can accelerate memory accesses if it can fetch larger chunks of memory than what the program might be currently accessing, in anticipation. This works remarkably well, but only if the program has some sense of locality. That is, if the program isn’t reading randomly from the memory. But from the hardware’s perspective, it sees both the host OS and guest OS and all their processes, which starts to approximate a Gaussian distribution when they’re all running in tandem, and that deeply impacts caching performance.

The hardware solution is to introduce an MMU that is amenable to virtualization, one which can manage both the host OS’s paged+virtual memory as well as any guest OS’s paged+virtual memory. Generally, this is known as Second Level Address Translation (SLAT) and is implemented as AMD’s Rrapid Virtualizaion Indexing or Intel’s Extended Page Tables. This feature allows the MMU to consider page tables – the basic unit of any MMU – that nest below a superior page table. In this way, the host OS can delegate to the guest a range of pages, and the guest OS can manage those pages, all while the MMU gives the guest OS some acceleration because this is all done in hardware.

This also helps with the caching situation, since if the MMU is aware that the memory is in a nested page table (ie guest OS memory), then that likely also means the existing cache for the host is irrelevant, and vice-versa. An optimization would be to split the cache space, so that it remains relevant only to the host or to the guest, without mixing up the two.

With all that said, we can now answer your question about what would happen .With hardware extensions like VT-x and SLAT, I would expect that cascading VMs would consume CPU and memory resources almost linearly, due to each guest OS adding its own overhead and running its own kernel. At some point, the memory performance would slow to a crawl, since there’s a limit on how much the physical cache can be split. But the CPU performance would likely be just fine, such as if you ran a calculation for digits of Pi on the 50th inner VM. Such calculations tend to use CPU registers rather than memory from DDR, and so could run natively on the CPU without trapping to any of the guests.

But I like the other commenter’s idea of just trying it and see what happens.