Porting UNIX to the 386:Porting Unix to the 386: Three Initial PC Utilities
The second article in the "PORTING UNIX TO THE 386" series discussed the utilities we had to build to test the port on an actual 80386 PC.
Dealing with PC reverse compatibility requires defeating logic like Gate A20, which forces address rollover to a single megabyte for ancient 8086/8088 compatibility.Consistency Checks
Its not enough just to load the kernel and go, you need to insure its the kernel bit-for-bit by the time you run it.The First PC Utility: boot.exe
Requirements for a DOS PC Utility to boot a 386BSD UNIX kernel.The GCC Executable Format
Our kernel is a UNIX executable file, just like all of its applications. So what does the a.out file, compiled and loaded usiing GCC and utilities look like?The Purpose of Our PC Utilities
Why we wrote PC utilities to port UNIX with - the advantage of working from a primitive OS that runs on the absolute machine.Unix Kernel Load Program
DOS C program for loading 386BSD kernel from DOS, and executing it, taking over the machine.Entering Protected Mode
After loading the kernel program, we enter protected mode to start up the 386BSD kernel, itself a protected mode program like its own applications.The Second PC Utility: cpfs.exe
First thing the kernel does is open its root filesystem - so we need a way to write a filesystem onto the hard disk, adjacent to the DOS filesystem, which is what this utility does.The Third PC Utility: cpsw.exe
Once the system is debugged and tested, the next step was to load on more code to expand with. So we moved "tar balls" to the swap space with this utility to provide a primative file upload capability.Where We Go From Here
Ironically, these utilities advanced progress fast enough that once the kernel was operational, the biggest obstacle became booting off of DOS. So the next step was to implement a bootstrap and standalone system so that we could rid ourself of DOS entirely.The 386BSD Project and Berkeley UNIX
Synopsis of what 386BSD was intended to be in the 1989-1990 timeframe.
This, the first article, is the first published mention of 386BSD. By this time, the project had been operational for 18 months, and William Jolitz was at Berkeley working on the Net/2 release.
We'd gathered books and equipment to begin the port in 1989. Most critical was the Crawford and Gelsinger book.
The 386BSD specification was in two parts - one that detailed getting to a operational system that could build itself and basic console applications, and a more extensive community involvement part, called "A Modest Proposal".The Definition of the 386BSD Specification
Choosing how far to go in supporting X86 architecture in order to get a basic BSD UNIX system to be able to bootstrap the futre efforts.Conflicts in Priorities
We resolve conflicts between UNIX worlds by choosing a middle way - one that isn't a pure standard, but one that doesn't fight standards commonality.386BSD Port Goals: A Practical Approach
We decided to shoot for a system that emphasized novel 386 support code to create a basic BSD environment on the 386. We assumed that by making it freely available, others will extend the work to remaining areas.386 Memory Management Vitals
Most popular microprocessors use either segmentation or paging to manage memory address space access. The 386 is rare in that it possesses both. In fact, since segmentation, (see Figure 3(a)), is placed on top of paging (see Figure 3(b)), you are expected to use segmentation in some form any time memory is paged. And, most important, BSD relies on paging.Segmentation and 386BSD
Reconciling segmentation to UNIX has never been easy, and with 386BSD its an even greater chore. The issues of supporting X86 segments in a Berkeley UNIX world.Kernel Linear Address Space Overhead
386BSD ignored as much as possible the segmentation hardware of the x86, preferring to use a "flat" address space.Virtual Address Space Layout
386BSD executed programs with a virtual address similar to a VAX running UNIX. In February 1991, this changed so that the format of the process virtual address space memory usage could be arranged differently.Per-Process Data Structures
A UNIX legacy, the "u." or per-process data structure, which held the kernel-related data of a process, was present on 386BSD prior to February 1991.386 Virtual Memory Address Translation Mechanism
The 386 Paging Memory Management allocates memory in 4KB and 4MB allocation units. This impacts the way programs execute and share file data.User to Kernel Communication Primitives
One surprise with 386BSD was that the 80386 doesn't honor write protection on the user page addresses, requiring a work-around. This was fixed in 80486 and all subsequent X86 processors.Berkeley UNIX Virtual Memory System Strategy
386BSD started out in 1989 with a derivative virtual memory system from the VAX by way of a 68030. In February/March 1991, it was cutover to a totally different one cut out of CMU's MACH system, and released with Net/2.Structure of Per-Process Data (u.)
The "u." in more detail, handling kernel stack overflows in 386BSD.Process Context Description
Hardware context switch state description and the part where 386BSD context switching intrudes into the machine independent code semantics.Other Processor Faults
Catching terminal processor faults.Microprocessor Idiosyncrasies
Sometimes you're forced to use processor features - like hardware context switching. Origionally, the earliest versions of 386BSD didn't use the hardware context switch TSS feature - but you still had to have one anyways.System Call Interface
How to call the system's kernel from a process, using existing industry standards accross the X86 platform.System Specific (ISA) Issues
The AT platform itself requires support in order to have an operational system. And with growth of elovling industry standards, this morphs over time into newer ones as well.Physical Memory Map
The actual memory referenced on processor, system, and I/O device busses, or physical memory addresses have standard assignments for the kernel to manage for the operating system to function.ISA Device Controllers
ISA devices that attach to the AT standard through controllers field a broad number of devices various categories.ISA Device Auto Configuration
To find controllers and devices, we use tables to instruct device drivers where to probe and attach found peripherals and connect them with low-level drivers and high-level kernel subsystems.Interrupt Priority Level Management
Part of the specification also details the interrupt control mechanism that allows device interrupts to be blocked/unblocked and caught by device drivers interrupt handling code.Bootstrap Operation
How to bootstrap the system from hardware, loading the kernel program, itself a protected mode executable from secondary / nonvolatile / disk storage.Summary: Where is 386BSD Now?
The state of the 386BSD world back in mid-1990 is synopsized here. By the time this appeared, EISA and other "beyond AT" support had been added. CSRG only let other UC institutions have code, although a complete binary and source release was present and tested for 6 months.Suggested Readings Microprocessor and System Specification Issues
Support the processor, and support the ISA bus peripherals are the objectives for the first parts of 386BSD.Page Fault and Segmentation Fault Mechanism
Catching potentially restartable 386 processor faults with 386BSD.Porting Unix to the 386: The Standalone System
This article, last of the original three done altogether in 1990, on getting the critical pieces functioning independantly that we needed to do the port. Once these we obtained, the kernel was inevitable.Watching for Land Mines
Anticipating problems allowed us to find flaws in our work. We use the standalone system for bootstrap to load test programs that work machine-dependant portions of the kernel.The First Step
We stepwise proved out the running environment of program tools, program loading and execution, trap handling, stack, and support of high level language. Our project moved from fragile to substantive.Introducing the Standalone System
Originally the test framework to prove kernel parts work in the environment of an protected mode program running on the "raw" machine, this became the bootstrap environment to later load and start the kernel in system operation.Standalone Keyboard Driver
Since we had a program running in an empty PC, we needed primative input with a keyboard driver for polled input. Used by standalone programs to boot and test parts of the kernel.Standalone Display Adapter Driver
Output from our standalone programs was done by a primative display driver to allow standalone programs to display results from bootstrap and test programs. We avoided the BIOS entirely.Prevaricating with the Standalone System
We didn't just load and debug the kernel; we chose to prove portions first. That way we learned the dependances first, and could try alternatives seperately. Later we used the same means to revise them later.Extending the Standalone System
Booting a kernel didn't require all of this - but by extending support, we had a "mini kernel" like functionality. Dillemma - how much do you let the boostrap/loader actually do? We chose after the kernel was up to have it to the most - but this answer is subject to review.Processor Support -- i386.c
We initialized the processor with initial descriptor and page tables - one needs to run with the tables before activating memory/interrupt kernel functions.Initial Task State Load
Here's an oddity we had in the initial port. We first avoided the 386 TSS context switch because it was slow. With the 486 and later, it wasn't so we used it. But we needed at lead one to transit rings.Trap Handling
We coded stub routines to connect trap handlers to descriptor table entry points, wiring the kernel to receive processor traps.Interrupt Handling
We coded specialized trap handlers for interrupt traps, as these were initimate with the systems interrupt control hardware, not part of the processor.System Call Handling
System calls were handled by a call gate, a 386 peculiar mechanism which allowed for ring crossing into supervisor mode using a special segment descriptor call. We simulated user mode to test prior to running in the kernel.Page Fault Handling
We coded trap handlers and simulated read and write faults to test out page faults with our processor support code. On a 2MB machine, we knew we'd get 1,000's.Where Do We Go From Here?
We had put the plan of the first article into action, used the second articles tools to load test programs as the kernel, extending standalone operation. This created a base for the kernel and user environment.Porting Unix to the 386: Language Tools Cross Support
We describe the need and use of a cross-support environment to create 386 code from a non-386 machine, so as to create the initial binarys before our port can generate them.Why Develop Cross-Tools?
We used another proven UNIX system - a Symmtric 375, to cross compile 386 code (bootstrap, kernel, shell, utilities) to get the first system running.386BSD Cross-Tools Goals
Our use of cross-tools was simply to bootstrap onto the 386 - not to perfect ongoing development. So, if we found a problem with the tool, we bypassed it with a one time workaround - cross tools didn't need to work for everything.What's in the Tool Chest?
The tool chest for 386BSD cross support included compiler, assembler, loader, libraries and include files. It did not include an emulation environment.Cross-Support Methodology
Our methodology was to prove that we could get a usable, tested executable across onto the native machine to be useful there. As we found and fixed, this methodology sped getting enough good and working native components, such that we could begin native development.Which C Standard?
BSD started before ANSI C, and still seems wedded to it even in 2006! GCC with the "traditional" flag didn't quite work, so we compromised.Other Cross-Support Issues
We started with a cheap lunchbox PC with a 40MB drive, and had to change as we exhausted serial downloads and DOS was squeezed of the drive for space for BSD.Validating GCC for Use in a Cross-Environment GCC Support Calls to Replace GNULIB
We used GCC but not GNULIB, so we had to roll our own support code as needed.Choosing a Sensible Cross-Host
Our Symmetric 375 was close but not identical to a 386 - its National 32000 series microprocessor was different. Yet it was a good choice for cross host in a cross development environment.Where Do We Go From Here
With cross tools we could make utility programs for our nascent system. The next step would be incorporating them into a filesystem so that they could run on the native 386, with the kernel program.Porting Unix to the 386: Copyrights, Copylefts, and Competitive Advantage
We describe the origin and orientation of the "Free Software" vs. "Open Software" efforts via respective licenses.Open Standards - Are they really "Open"?
Open standards, open source, and free software are not the same thing - they are very different.Berkeley Copyright License
386BSD is a Berkeley system, its authors were from Berkeley, and it is under a Berkeley license.Copyright vs. Copyleft
The view of copyright and the GPL as it was back in the early 90's.The Role of the Root Filesystem
The root filesystem is a small, essential portion of disk storage, providing functionality to expand its resources to use storage other than the root itself, and configure system operations. The root usually survives intact when a system crash occurs, allowing system operation during recovery.A Brief Review of the Root
A breakdown of the various uses of the root filesystems, and the considerations for each as we prove out the operation of the system step by step.Installation: /stand
Portions of the root allow the kernel to be installed as the fundamental component of the operating system. These standalone programs occupy space within the root filesystem.Initialization: /sbin/init, /dev/console, and /bin/sh
With the kernel running, we do a "high level bootstrap" to initialize the operating system, by executing certain programs located in the root filesystem.Utilities: /bin and /sbin
A basic set of utilities is present on the root, both of general user operation as well as necessary supervisory utilities for system recovery and operation.Operation: /tmp and /var
In operation, directories in the root are reserved for temporary and persistant data, such as compiler temporary files and accounting records.Other Directories: /lib, /mnt, /usr, /root, and /sys
A number of additional directorys are present in the root to handle a variety of other purposes, but these are more to do with full operation of the system beyond the initial root filesystem.Filesystem Creation
Where do you get the root filesystem, when you need it to make itself? Answer - you make it on another system, in this case one quite different, and then you break the recursion.Filesystem Downloading
Having made the initial filesystem on another kind of system, we need to move it into place on the system we are running the kernel on.Filesystem Debugging
Nothing ever goes right the first time, so a incremental process of bringing up the filesystem, from standalone utilities to system initialization allows us to debug flaws in filesystem creation, often artifacts of its non-native creation.What's in a Filesystem?
What is it that makes up a filesystem? This depends very much on what kind of filesystem we are talking about.Why Do We Need a Root Filesystem?
Whats different about operating systems like Unix that use a root filesystem, and other systems that don't require a filesystem to be mounted initially to operate?The Filesystem Metaphor and its Importance in Future Work
Filesystem as metaphor allows us to restructure the way we organize system operation - perhaps other arrangements of the filesystem can achieve more enhanced operation, possibly rethinking how to bring up the system in the future.Porting Unix to the 386: Research and the Commercial Sector
Understanding the boundary between research and development with BSD, and where a balance between commercial efforts can be struck.