This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
book-summary:unix-haters-handbook [2017/02/02 01:33] admin |
book-summary:unix-haters-handbook [2020/06/13 20:09] (current) |
||
|---|---|---|---|
| Line 38: | Line 38: | ||
| <blockquote> | <blockquote> | ||
| - | The wonderful thing about standards is that there are so many of them to choose from</blockquote>. | + | The wonderful thing about standards is that there are so many of them to choose from</blockquote> |
| What sort of specification does a version of Unix satisfy? POSIX? X/Open? CORBA? There is so much wiggle room in these standards as to make the idea that a company might have liability for not following them ludicrous to ponder. Indeed, everybody follows these self-designed standards, yet none of the products are compatible. | What sort of specification does a version of Unix satisfy? POSIX? X/Open? CORBA? There is so much wiggle room in these standards as to make the idea that a company might have liability for not following them ludicrous to ponder. Indeed, everybody follows these self-designed standards, yet none of the products are compatible. | ||
| Line 271: | Line 271: | ||
| To: <PAGANISM Digest Subscriber> | To: <PAGANISM Digest Subscriber> | ||
| - | This message was sent to PAGANISM-REQUEST, not PAGANISM. Either you or your 'r' key | + | This message was sent to PAGANISM-REQUEST, not PAGANISM. Either you or your 'r' key screwed up here. Or else the digest is screwed up. Anyway, you could try sending it again. |
| - | screwed up here. Or else the digest is screwed up. Anyway, you could try sending it | + | -Devon |
| - | again. | + | |
| - | -Devon | + | |
| The clueless weenie sent back the following message to Devon, complaining that the fault lie not with himself or sendmail, but with the PAGANISM digest itself: | The clueless weenie sent back the following message to Devon, complaining that the fault lie not with himself or sendmail, but with the PAGANISM digest itself: | ||
| Line 282: | Line 280: | ||
| To: Devon Sean McCullough <devon@ghoti.lcs.mit.edu> | To: Devon Sean McCullough <devon@ghoti.lcs.mit.edu> | ||
| - | >From my perspective, the digest is at fault. Berkeley Unix Mail is what I use, and | + | >From my perspective, the digest is at fault. Berkeley Unix Mail is what I use, and |
| - | >it ignores the "Reply-to:" line, using the "From:" line instead. So the only way | + | >it ignores the "Reply-to:" line, using the "From:" line instead. So the only way |
| - | >for me to get the correct address is to either back-space over the dash and type | + | >for me to get the correct address is to either back-space over the dash and type |
| - | >the @ etc in, or save it somewhere and go trough some contortions to link the | + | >the @ etc in, or save it somewhere and go trough some contortions to link the |
| - | >edited file to the old echoed address. Why make me go to all that trouble? This is | + | >edited file to the old echoed address. Why make me go to all that trouble? This is |
| - | >the main reason that I rarely post to the PAGANISM digest at MIT. | + | >the main reason that I rarely post to the PAGANISM digest at MIT. |
| The interpretation of which is all too easy to understand: | The interpretation of which is all too easy to understand: | ||
| Line 296: | Line 294: | ||
| Subject: Depressing | Subject: Depressing | ||
| + | <blockquote> | ||
| Notice the typical Unix weenie reasoning here: | Notice the typical Unix weenie reasoning here: | ||
| - | "The digestifier produces a header with a proper Reply-to field, in the | + | The digestifier produces a header with a proper Reply-to field, in the |
| expectation that your mail reading tool will interpret the header in the documented, | expectation that your mail reading tool will interpret the header in the documented, | ||
| standard, RFC822 way. Berkeley Unix Mail, contrary to all standards, and unlike all | standard, RFC822 way. Berkeley Unix Mail, contrary to all standards, and unlike all | ||
| reasonable mail reading tools, ignores the Reply-To field and incorrectly uses the | reasonable mail reading tools, ignores the Reply-To field and incorrectly uses the | ||
| - | From field instead." | + | From field instead. |
| + | </blockquote> | ||
| Therefore: | Therefore: | ||
| Line 371: | Line 371: | ||
| ====== Part 7: The X-Windows disaster ====== | ====== Part 7: The X-Windows disaster ====== | ||
| + | <blockquote> | ||
| + | How to make a 50-MIPS workstation run like a 4.77 MHz IBM PC</blockquote> | ||
| - | "How to make a 50-MIPS workstation run like a 4.77 MHz IBM PC" | + | <blockquote>If the designer of X Windows built cars, there would be no fewer than five steering wheels hidden about the cockpit, none of which followed the same principles - but you'd be able to shift gears with your car stereo. Useful feature, that.</blockquote> |
| - | "If the designer of X Windows built cars, there would be no fewer than five steering wheels hidden about the cockpit, none of which followed the same principles - but you'd be able to shift gears with your car stereo. Useful feature, that." -Marcus J. Ranum, Digital Equipment Corporation. | + | -Marcus J. Ranum, Digital Equipment Corporation. |
| X windows is the Iran-Contra of graphical user interfaces: a tragedy of political compromises, entangled alliances, marketing hype, and just plain greed. X windows is to memory as Ronald Reagan was to money. Years of "Voodoo Ergonomics" have resulted in an unprecedented memory deficit of gargantuan proportions. Divisive dependencies, distributed deadlocks, and partisan protocols have tightened gridlocks, aggravated race conditions, and promulgated double standards. | X windows is the Iran-Contra of graphical user interfaces: a tragedy of political compromises, entangled alliances, marketing hype, and just plain greed. X windows is to memory as Ronald Reagan was to money. Years of "Voodoo Ergonomics" have resulted in an unprecedented memory deficit of gargantuan proportions. Divisive dependencies, distributed deadlocks, and partisan protocols have tightened gridlocks, aggravated race conditions, and promulgated double standards. | ||
| Line 380: | Line 382: | ||
| X has had its share of $5,000 toilet seats -like Sun's Open Look clock tool, which gobbles up 1.4 megabytes of real memory! If you sacrificed all the RAM from 22 Commodore 64s to clock tool, it still wouldn't have enough to tell you the time. Even the vanilla X11R4 "xclock" utility consumes 656 to run. And X's memory usage is increasing. | X has had its share of $5,000 toilet seats -like Sun's Open Look clock tool, which gobbles up 1.4 megabytes of real memory! If you sacrificed all the RAM from 22 Commodore 64s to clock tool, it still wouldn't have enough to tell you the time. Even the vanilla X11R4 "xclock" utility consumes 656 to run. And X's memory usage is increasing. | ||
| - | ==== X: The first fully modular software disaster ==== | + | ===== X: The first fully modular software disaster ===== |
| X Windows started out as one man's project in an office on the fifth floor of MIT's laboratory for computer science. A wizardly hacker, who was familiar with W, a window system written at Stanford University as part of the V project, decided to write a distributed graphical display server. The idea was to allow a program, called a client, to run on one computer and allow it to display on another computer that was running a special program called a window server. The two computer might be VAXes or Suns, or one of each, as long as the computers were networked together and each implemented the X protocol. | X Windows started out as one man's project in an office on the fifth floor of MIT's laboratory for computer science. A wizardly hacker, who was familiar with W, a window system written at Stanford University as part of the V project, decided to write a distributed graphical display server. The idea was to allow a program, called a client, to run on one computer and allow it to display on another computer that was running a special program called a window server. The two computer might be VAXes or Suns, or one of each, as long as the computers were networked together and each implemented the X protocol. | ||
| Line 386: | Line 388: | ||
| Note: We have tried to avoid paragraph-length footnotes in this book, but X has defeated us by switching the meaning of client and server. In all other client/server relationships, the server is the remote machine that runs the application (i.e., the server provides services, such as database service or computation service). For some perverse reason that's better left to imagination, X insists on calling the program running on the remote machine "the client". This program displays its windows on the "window server". We're going to follow X terminology when discussing graphical client/servers. So when you see "client" think "the remote machine where the application is running" and when you see "server" think "the local machine that displays output and accepts user input." | Note: We have tried to avoid paragraph-length footnotes in this book, but X has defeated us by switching the meaning of client and server. In all other client/server relationships, the server is the remote machine that runs the application (i.e., the server provides services, such as database service or computation service). For some perverse reason that's better left to imagination, X insists on calling the program running on the remote machine "the client". This program displays its windows on the "window server". We're going to follow X terminology when discussing graphical client/servers. So when you see "client" think "the remote machine where the application is running" and when you see "server" think "the local machine that displays output and accepts user input." | ||
| - | ==== The nongraphical GUI ==== | + | ===== The nongraphical GUI ===== |
| X was designed to run three programs: xterm, xload, and xclock. (The idea of a window manager was added as an afterthought, and it shows.) For the first few years of its development at MIT, these were, in fact, the only programs that ran under the window system. Notice that none of these programs have any semblance of a graphical user interface (except xclock), only one of these programs implements anything in the way of cut-and-paste (and then, only a single data type is supported), and none of them requires a particularly sophisticated approach to color management. Is it any wonder, then, that these all areas in which moder X falls down? Ten years later, most computers running X run just four programs: xterm, xload, xclock, and a window manager. And most xterm windows run Emacs! X has to be the most expensive way ever of popping up an Emacs window. It sure would have been much cheaper and easier to put terminal handling in the kernel where it belongs, rather that forcing people to purchase expensive bit mapped terminals to run character-based applications. On the other hand, then users wouldn't get all of those ugly fonts. It's a trade-off. | X was designed to run three programs: xterm, xload, and xclock. (The idea of a window manager was added as an afterthought, and it shows.) For the first few years of its development at MIT, these were, in fact, the only programs that ran under the window system. Notice that none of these programs have any semblance of a graphical user interface (except xclock), only one of these programs implements anything in the way of cut-and-paste (and then, only a single data type is supported), and none of them requires a particularly sophisticated approach to color management. Is it any wonder, then, that these all areas in which moder X falls down? Ten years later, most computers running X run just four programs: xterm, xload, xclock, and a window manager. And most xterm windows run Emacs! X has to be the most expensive way ever of popping up an Emacs window. It sure would have been much cheaper and easier to put terminal handling in the kernel where it belongs, rather that forcing people to purchase expensive bit mapped terminals to run character-based applications. On the other hand, then users wouldn't get all of those ugly fonts. It's a trade-off. | ||
| - | ==== The Motif self-abuse kit ==== | + | ===== The Motif self-abuse kit ===== |
| X gave Unix vendors something they had professed to want for years: a standard that allowed programs built for different computers to interpreted. But it didn't give them enough. X gave programmers a way to display windows and pixels, but it didn't speak to buttons, menus, scroll bars, or any of the other necessary elements of a graphical user interface. Programmers invented their own. Soon the Unix community had six or so different interface standards. A bunch of people who hadn't written 10 lines of code in as many years set up shop in a brick building in Cambridge, Massachusetts, that was the former house of a failed computer company and came up with a "solution" the Open Software Foundation's Motif. | X gave Unix vendors something they had professed to want for years: a standard that allowed programs built for different computers to interpreted. But it didn't give them enough. X gave programmers a way to display windows and pixels, but it didn't speak to buttons, menus, scroll bars, or any of the other necessary elements of a graphical user interface. Programmers invented their own. Soon the Unix community had six or so different interface standards. A bunch of people who hadn't written 10 lines of code in as many years set up shop in a brick building in Cambridge, Massachusetts, that was the former house of a failed computer company and came up with a "solution" the Open Software Foundation's Motif. | ||
| Line 398: | Line 400: | ||
| Recipe for disaster: start with the Microsoft Windows metaphor, which was designed and hand coded in assembler. Build something on top of three or four layers of X to look like Windows. Call it "Motif". Now put two 486 boxes side by side, one running Windows and one running Unix/Motif. Watch one crawl. Watch it wither. Watch it drop faster than the putsh in Russia. Motif can't compete with the Macintosh OS or with DOS/Windows as a delivery platform. | Recipe for disaster: start with the Microsoft Windows metaphor, which was designed and hand coded in assembler. Build something on top of three or four layers of X to look like Windows. Call it "Motif". Now put two 486 boxes side by side, one running Windows and one running Unix/Motif. Watch one crawl. Watch it wither. Watch it drop faster than the putsh in Russia. Motif can't compete with the Macintosh OS or with DOS/Windows as a delivery platform. | ||
| - | ==== Ice cube: The lethal weapon ==== | + | ===== Ice cube: The lethal weapon ===== |
| One of the fundamental design goals of X was to separate the window manager from the window server. "Mechanism, not policy" was the mantra. That is, the X servers provided a mechanism for drawing on the screen and managing windows, but did not implement a particular policy for human-computer interaction. While this might have seemed like a good idea at the time (especially if you are in a research community, experimenting with different approaches for solving the human-computer interaction problem), it created a veritable user interface Tower of Babel. | One of the fundamental design goals of X was to separate the window manager from the window server. "Mechanism, not policy" was the mantra. That is, the X servers provided a mechanism for drawing on the screen and managing windows, but did not implement a particular policy for human-computer interaction. While this might have seemed like a good idea at the time (especially if you are in a research community, experimenting with different approaches for solving the human-computer interaction problem), it created a veritable user interface Tower of Babel. | ||
| Line 410: | Line 412: | ||
| In summary, ICCCM is a technological disaster: a toxic waste dump of broken protocols, backward compatibility nightmares, complex non solutions to obsolete non problems, a twisted mass of scabs and scar tissue intended to cover up the moral and intellectual depravity of the industry's standard naked emperor. | In summary, ICCCM is a technological disaster: a toxic waste dump of broken protocols, backward compatibility nightmares, complex non solutions to obsolete non problems, a twisted mass of scabs and scar tissue intended to cover up the moral and intellectual depravity of the industry's standard naked emperor. | ||
| - | ==== X Myths ==== | + | ===== X Myths ===== |
| * Myth: X demonstrates the power of client/server computing | * Myth: X demonstrates the power of client/server computing | ||
| Line 427: | Line 429: | ||
| ====== Part 8: csh, pipes, and find ====== | ====== Part 8: csh, pipes, and find ====== | ||
| - | "Power tools for power fools" | + | <blockquote>Power tools for power fools</blockquote> |
| - | "I have a natural revulsion to any operating system that shows so little planning as | + | <blockquote> |
| + | I have a natural revulsion to any operating system that shows so little planning as | ||
| to have to named all of its commands after digestive noises (awk, grep, fsck, | to have to named all of its commands after digestive noises (awk, grep, fsck, | ||
| - | nroff)." -Unknown. | + | nroff).</blockquote> |
| The Unix "power tool" metaphor is a canard. It's nothing more than a slogan behind which Unix hides its arcane patchwork of commands and adhoc utilities. A real power tool amplifies the power of its user with little additional effort or instruction. Anyone capable of using screwdriver or drill can use a power screwdriver or power drill. The user needs no understanding of electricity, motors, torquing, magnetism, heat dissipation, or maintenance. She just needs to plug it in, wear safety glasses. It's rare to find a power tool that is fatally flawed in the hardware store: most badly designed power tools either don't make it to market or result in costly lawsuits, removing them from the market and punishing their makers. | The Unix "power tool" metaphor is a canard. It's nothing more than a slogan behind which Unix hides its arcane patchwork of commands and adhoc utilities. A real power tool amplifies the power of its user with little additional effort or instruction. Anyone capable of using screwdriver or drill can use a power screwdriver or power drill. The user needs no understanding of electricity, motors, torquing, magnetism, heat dissipation, or maintenance. She just needs to plug it in, wear safety glasses. It's rare to find a power tool that is fatally flawed in the hardware store: most badly designed power tools either don't make it to market or result in costly lawsuits, removing them from the market and punishing their makers. | ||
| Line 437: | Line 440: | ||
| Unix power tools don't fit this mold. Unlike the modest goals of its designers to have tools that were simple and single-purposed, today's Unix tools are over-featured, over-designed, and over-engineered. For example, ls, a program that once only listed files, now has more that 18 different options that control everything from sort order to the number of columns in which the printout appears -all functions that are better handled with other tools (and once were). The find command writes cpio-formatted output files in addition to finding files (something easily done by connecting the two command with an infamous Unix pipe). Today, the Unix equivalent of a power drill would have 20 dials and switches, come with a nonstandard plug, require the user to hand-wind the motor coil, and not accept 3/8" or 7/8" drill bits (though this would be documented in the BUGS section of its instruction manual). | Unix power tools don't fit this mold. Unlike the modest goals of its designers to have tools that were simple and single-purposed, today's Unix tools are over-featured, over-designed, and over-engineered. For example, ls, a program that once only listed files, now has more that 18 different options that control everything from sort order to the number of columns in which the printout appears -all functions that are better handled with other tools (and once were). The find command writes cpio-formatted output files in addition to finding files (something easily done by connecting the two command with an infamous Unix pipe). Today, the Unix equivalent of a power drill would have 20 dials and switches, come with a nonstandard plug, require the user to hand-wind the motor coil, and not accept 3/8" or 7/8" drill bits (though this would be documented in the BUGS section of its instruction manual). | ||
| - | ==== The shell game ==== | + | ===== The shell game ===== |
| The inventors of Unix had a great idea: make the command processor be just another user-level program. If users didn't like the default command processor, they could write their own. More importantly, shells could evolve, presumably so that they could become more powerful, flexible, and easy to use. | The inventors of Unix had a great idea: make the command processor be just another user-level program. If users didn't like the default command processor, they could write their own. More importantly, shells could evolve, presumably so that they could become more powerful, flexible, and easy to use. | ||
| Line 453: | Line 456: | ||
| * bash The GNU bourne-again shell | * bash The GNU bourne-again shell | ||
| - | ==== Pipes ==== | + | ===== Pipes ===== |
| Unix lovers believe in the purity, virtue, and beauty of pipes. They extol pipes as the mechanism that, more that any other feature, makes Unix Unix. "Pipes", Unix lovers intone over and over again, "allow complex programs to be built out of simpler programs. Pipes allow programs to be used in unplanned and unanticipated ways. Pipes allow simple implementations." Unfortunately, chanting mantras doesn't do Unix any more good that it does the Hari Krishnas. | Unix lovers believe in the purity, virtue, and beauty of pipes. They extol pipes as the mechanism that, more that any other feature, makes Unix Unix. "Pipes", Unix lovers intone over and over again, "allow complex programs to be built out of simpler programs. Pipes allow programs to be used in unplanned and unanticipated ways. Pipes allow simple implementations." Unfortunately, chanting mantras doesn't do Unix any more good that it does the Hari Krishnas. | ||
| Line 471: | Line 474: | ||
| Pipes are not be-all and end-all of program communication. Our favorite Unix-loving book had this to say about the Macintosh, which doesn't have pipes: | Pipes are not be-all and end-all of program communication. Our favorite Unix-loving book had this to say about the Macintosh, which doesn't have pipes: | ||
| - | "The Macintosh model, on the other hand, is the exact opposite. The system doesn't deal with character streams. Data files are extremely high level, usually assuming that they are specific to an application. When was the last time you piped the output of one program to another on a Mac? (Good lock even finding the pipe symbol.) Programs are monolithic, the better to completely understand what you are doing. You don't take MacFoo and MacBar and hook them together." -From life with Unix, by Libes and Ressler | + | <blockquote>The Macintosh model, on the other hand, is the exact opposite. The system doesn't deal with character streams. Data files are extremely high level, usually assuming that they are specific to an application. When was the last time you piped the output of one program to another on a Mac? (Good lock even finding the pipe symbol.) Programs are monolithic, the better to completely understand what you are doing. You don't take MacFoo and MacBar and hook them together.</blockquote> |
| + | |||
| + | - From life with Unix, by Libes and Ressler | ||
| Yeah, those poor Mac users. They've got it so rough. Because they can't pipe streams of bytes around into their latest memo and gave text flow around it? How are they going to transfer a spreadsheet into their memo? And how could such users expect changes to be tracked automatically? Theu certainly shouldn't expect to be able to electronically mail this patched-together memo across the country and have it seamlessly read and edited at the other end, and the returned to them unscathed. We can't imagine how they've been transparently using all these programs together for the last 10 years and having them all work, all without pipes. | Yeah, those poor Mac users. They've got it so rough. Because they can't pipe streams of bytes around into their latest memo and gave text flow around it? How are they going to transfer a spreadsheet into their memo? And how could such users expect changes to be tracked automatically? Theu certainly shouldn't expect to be able to electronically mail this patched-together memo across the country and have it seamlessly read and edited at the other end, and the returned to them unscathed. We can't imagine how they've been transparently using all these programs together for the last 10 years and having them all work, all without pipes. | ||
| Line 477: | Line 482: | ||
| Research has shown that pipes and redirection are hard to use, not because of conceptual problems, but because of arbitrary and unintuitive limitations. It is documented that only those steeped in Unixdom, not run-of-the-mill users, can appreciate or use the power of pipes. | Research has shown that pipes and redirection are hard to use, not because of conceptual problems, but because of arbitrary and unintuitive limitations. It is documented that only those steeped in Unixdom, not run-of-the-mill users, can appreciate or use the power of pipes. | ||
| - | ==== Find ==== | + | ===== Find ===== |
| - | "The most horrifying thing about Unix is that, no matter how many times you hit yourself over the head with it, you never quite manage to lose consciousness. It just goes on and on." -Patrick Sobalvarro | + | <blockquote> |
| + | The most horrifying thing about Unix is that, no matter how many times you hit yourself over the head with it, you never quite manage to lose consciousness. It just goes on and on.</blockquote> | ||
| The Apple Macintosh and Microsoft Windows have powerful file locators that are relatively easy to use and extremely reliable. These file finders were designed with a human user and modern networking in mind. The Unix file finder program, find, wasn't designed to work with humans, but with cpio -a Unix backup utility program. Find couldn't anticipate networks or enhancements to the file system such as symbolic links; even after extensive modifications, it still doesn't work well with either. As a result, despite its importance to humans who've misplaced their files, find doesn't work reliably or predictably. | The Apple Macintosh and Microsoft Windows have powerful file locators that are relatively easy to use and extremely reliable. These file finders were designed with a human user and modern networking in mind. The Unix file finder program, find, wasn't designed to work with humans, but with cpio -a Unix backup utility program. Find couldn't anticipate networks or enhancements to the file system such as symbolic links; even after extensive modifications, it still doesn't work well with either. As a result, despite its importance to humans who've misplaced their files, find doesn't work reliably or predictably. | ||
| Line 487: | Line 493: | ||
| ====== Part 9: Programming ====== | ====== Part 9: Programming ====== | ||
| - | "Hold still, this won't hurt a bit" | + | <blockquote>Hold still, this won't hurt a bit</blockquote> |
| - | "Do not meddle in the affairs of Unix, for it is subtle and quick to core dump." -Anonymous. | + | <blockquote> |
| + | Do not meddle in the affairs of Unix, for it is subtle and quick to core dump.</blockquote> | ||
| - | ==== The wonderful Unix programming environment ==== | + | ===== The wonderful Unix programming environment ===== |
| The Unix zealots make much of the Unix "programming environment." They claim Unix has a rich set of tools that makes programming easier. Unix is not the world's best software environment -it is not even a good one. The Unix programming tools are shame; interpreters remain the play toy of the very rich; and change logs and audit trails are recorded at the whim of the person being audited. Yet somehow Unix maintains its reputation as a programmer's dream. Maybe it lets programmers dream about being productive, rather than letting them actually be productive. | The Unix zealots make much of the Unix "programming environment." They claim Unix has a rich set of tools that makes programming easier. Unix is not the world's best software environment -it is not even a good one. The Unix programming tools are shame; interpreters remain the play toy of the very rich; and change logs and audit trails are recorded at the whim of the person being audited. Yet somehow Unix maintains its reputation as a programmer's dream. Maybe it lets programmers dream about being productive, rather than letting them actually be productive. | ||
| - | ==== Don't know to make love, Stop ==== | + | ===== Don't know to make love, Stop ===== |
| The ideal programming tool should be quick and easy to use for common tasks and, at the same time, powerful enough to handle tasks beyond that for which it was intended. Unfortunately, in their zeal to be general, many Unix tools forget about the quick and easy part. Make is one such tool. In abstract terms, make's input is a description of a dependency graph. Each node of the dependency graph contains a set of commands to be run when that node is out of date with respect to the nodes that id depends on. Nodes corresponds to files, and the file dates determine whether the file are out of date with respect to each other. A small dependency graph, or Makefile, is shown below: | The ideal programming tool should be quick and easy to use for common tasks and, at the same time, powerful enough to handle tasks beyond that for which it was intended. Unfortunately, in their zeal to be general, many Unix tools forget about the quick and easy part. Make is one such tool. In abstract terms, make's input is a description of a dependency graph. Each node of the dependency graph contains a set of commands to be run when that node is out of date with respect to the nodes that id depends on. Nodes corresponds to files, and the file dates determine whether the file are out of date with respect to each other. A small dependency graph, or Makefile, is shown below: | ||
| Line 512: | Line 519: | ||
| While make's model is quite general, the designers forgot to make it easy to use for common cases. In fact, very few novices Unix programmers know exactly how utterly easy it is to screw yourself to a wall with make, until they do it. | While make's model is quite general, the designers forgot to make it easy to use for common cases. In fact, very few novices Unix programmers know exactly how utterly easy it is to screw yourself to a wall with make, until they do it. | ||
| - | ==== Utility programs and man pages ==== | + | ===== Utility programs and man pages ===== |
| Unix utilities are self-contained; each is free to interpret its command-line arguments as it sees fit. This freedom is annoying; instead of being able to learn a single set of conventions for command line arguments, you have to read a man page for each program to figure out how to use it. | Unix utilities are self-contained; each is free to interpret its command-line arguments as it sees fit. This freedom is annoying; instead of being able to learn a single set of conventions for command line arguments, you have to read a man page for each program to figure out how to use it. | ||
| Line 518: | Line 525: | ||
| The source is the documentation. Oh, great! | The source is the documentation. Oh, great! | ||
| - | "If it was hard to write, it should be hard to understand." -A Unix programmer | + | <blockquote> |
| + | If it was hard to write, it should be hard to understand.</blockquote> | ||
| + | - A Unix programmer | ||
| Back in the documentation chapter, we said that Unix programmers believe that the operating system's source code is the ultimate documentation. "After all", says one noted Unix historian, "the source is the documentation that the operating system itself looks to when it tries to figure out what to do next." | Back in the documentation chapter, we said that Unix programmers believe that the operating system's source code is the ultimate documentation. "After all", says one noted Unix historian, "the source is the documentation that the operating system itself looks to when it tries to figure out what to do next." | ||
| Line 532: | Line 541: | ||
| These mean that something is wrong. You should able to figure out exactly what it is that's wrong in each case. | These mean that something is wrong. You should able to figure out exactly what it is that's wrong in each case. | ||
| - | ==== It can't be a bug, my makefile depends on it! ==== | + | ===== It can't be a bug, my makefile depends on it! ===== |
| The programmers at BBN were generally the exception. Most Unix programmers don't fix bugs: most don't have source code. Those with the code know that fixing bugs won't help. That's why when most Unix programmers encounter a bug, they simply program around it. | The programmers at BBN were generally the exception. Most Unix programmers don't fix bugs: most don't have source code. Those with the code know that fixing bugs won't help. That's why when most Unix programmers encounter a bug, they simply program around it. | ||
| Line 542: | Line 551: | ||
| Unix follows the older "debugging as autopsy" model. In Unix, a broken program dies, leaving a core file, that is like a dead body in more ways than one. A Unix debugger then comes along and determines the cause of death. Interestingly enough, Unix programs tend to die from curable diseases, accidents, and negligence, just as people do. | Unix follows the older "debugging as autopsy" model. In Unix, a broken program dies, leaving a core file, that is like a dead body in more ways than one. A Unix debugger then comes along and determines the cause of death. Interestingly enough, Unix programs tend to die from curable diseases, accidents, and negligence, just as people do. | ||
| - | ==== Dealing with the Core ==== | + | ===== Dealing with the Core ===== |
| After your program has written out a core file, your fist task is to find it. This shouldn't be too difficult a task, because the core file is quite large -4,8 and even 12 megabytes core files are not uncommon. | After your program has written out a core file, your fist task is to find it. This shouldn't be too difficult a task, because the core file is quite large -4,8 and even 12 megabytes core files are not uncommon. | ||
| Line 550: | Line 559: | ||
| For instance, one cannot run a debugger as a command-interpreter or transfer control to debugger when the operating system generates an exception. The only way to have a debugger take over from your program when it crashes is to run every program from your debugger. | For instance, one cannot run a debugger as a command-interpreter or transfer control to debugger when the operating system generates an exception. The only way to have a debugger take over from your program when it crashes is to run every program from your debugger. | ||
| - | ==== Filename expansion ==== | + | ===== Filename expansion ===== |
| There is one exception to Unix's each-program-is-self-contained rule: file-name expansion. Very often, one wants Unix utilities to operate on one or more files. The Unix shells provide a shorthand for naming groups of files that are expanded by the shell, producing a list of files that is passed to the utility. | There is one exception to Unix's each-program-is-self-contained rule: file-name expansion. Very often, one wants Unix utilities to operate on one or more files. The Unix shells provide a shorthand for naming groups of files that are expanded by the shell, producing a list of files that is passed to the utility. | ||
| Line 556: | Line 565: | ||
| For example, say your directory contains the files A, B, and C. To remove all of these files, you might type 'rm *'. The shell will expand '*' to 'A B C' and pass these arguments to rm. There are many, many problems with this approach, which we discussed in the previous chapter. You should know, though, that using the shell to expand filenames is not an historical accident: it was a carefully reasoned designed decision. In "The Unix programming environment" by Kernighan and Mashey (IEEE Computer, April 1981), the authors claim that, "Incorporating this mechanism into the shell is more efficient that duplicating it everywhere and ensures that it is available to programs in a uniform way" | For example, say your directory contains the files A, B, and C. To remove all of these files, you might type 'rm *'. The shell will expand '*' to 'A B C' and pass these arguments to rm. There are many, many problems with this approach, which we discussed in the previous chapter. You should know, though, that using the shell to expand filenames is not an historical accident: it was a carefully reasoned designed decision. In "The Unix programming environment" by Kernighan and Mashey (IEEE Computer, April 1981), the authors claim that, "Incorporating this mechanism into the shell is more efficient that duplicating it everywhere and ensures that it is available to programs in a uniform way" | ||
| - | ==== Robustness, or "All lines are shorter than 80 characters" ==== | + | ===== Robustness, or "All lines are shorter than 80 characters" ===== |
| There is an amusing article in the December 1990 issue of "Communications of the ACM" entitled "An empirical study of the reliability of Unix utilities" by Miller, Fredriksen, and So. They fed random input to a number of Unix utility programs and found that they could make 24-33% (depending on which vendor's Unix was being tested) of the programs crash or hang. Occasionally the entire operating system panicked. | There is an amusing article in the December 1990 issue of "Communications of the ACM" entitled "An empirical study of the reliability of Unix utilities" by Miller, Fredriksen, and So. They fed random input to a number of Unix utility programs and found that they could make 24-33% (depending on which vendor's Unix was being tested) of the programs crash or hang. Occasionally the entire operating system panicked. | ||
| Line 583: | Line 592: | ||
| str[3] == 'y' | str[3] == 'y' | ||
| - | ==== Isn't C grand? ==== | + | ===== Isn't C grand? ===== |
| The problem with this approach is that C doesn't do any automatic bounds checking on the array references. Why should it? The arrays are really just pointer, and you can have pointers to anywhere in memory, right? Well, you might want to ensure that a piece of code doesn't scribble all over arbitrary pieces of memory, especially if the piece of memory in question is important, like the program's stack. | The problem with this approach is that C doesn't do any automatic bounds checking on the array references. Why should it? The arrays are really just pointer, and you can have pointers to anywhere in memory, right? Well, you might want to ensure that a piece of code doesn't scribble all over arbitrary pieces of memory, especially if the piece of memory in question is important, like the program's stack. | ||
| Line 610: | Line 619: | ||
| We say "probably" because you can corrupt the runtime stack to achieve an effect that the original programmer never intended. Imagine that out function was called upon to read a really long line, over 2,000 characters, and that this line was set up to overwrite the bookkeeping information on the call stack so that when the C function returns, it will call a piece of code that was also embedded in the 2.000 character line. This embedded piece of code may do something truly useful, like exec a shell that can run command on the machine. | We say "probably" because you can corrupt the runtime stack to achieve an effect that the original programmer never intended. Imagine that out function was called upon to read a really long line, over 2,000 characters, and that this line was set up to overwrite the bookkeeping information on the call stack so that when the C function returns, it will call a piece of code that was also embedded in the 2.000 character line. This embedded piece of code may do something truly useful, like exec a shell that can run command on the machine. | ||
| - | ==== Exceptional conditions ==== | + | ===== Exceptional conditions ===== |
| The main challenge of writing robust software is gracefully handling errors and other exceptions. Unfortunately, C provides almost no support for handling exceptional conditions. As a result, few people learning programming in today's school and universities know what exceptions are. | The main challenge of writing robust software is gracefully handling errors and other exceptions. Unfortunately, C provides almost no support for handling exceptional conditions. As a result, few people learning programming in today's school and universities know what exceptions are. | ||
| Line 649: | Line 658: | ||
| Lisp implementations usually have real exception-handling systems. The exceptional conditions have names like OUT-OF-MEMORY and the programmer can establish exception handlers for specific types of conditions. These handlers get called automatically when the exception are raised -no intervention or special tests are needed on the part of the programmer. When used properly, these handlers lead to more robust software. | Lisp implementations usually have real exception-handling systems. The exceptional conditions have names like OUT-OF-MEMORY and the programmer can establish exception handlers for specific types of conditions. These handlers get called automatically when the exception are raised -no intervention or special tests are needed on the part of the programmer. When used properly, these handlers lead to more robust software. | ||
| - | ==== If you can't fix it, restart it! ==== | + | ===== If you can't fix it, restart it! ===== |
| So what do system administrators and others do with vital software that doesn't properly handle errors, bad data, and bad operating conditions? Well, if it runs OK for a short period of time, you can make it run for a long period of time by periodically restarting it. The solution isn't very reliable, nor scalable, but it is good enough to keep Unix creaking along. | So what do system administrators and others do with vital software that doesn't properly handle errors, bad data, and bad operating conditions? Well, if it runs OK for a short period of time, you can make it run for a long period of time by periodically restarting it. The solution isn't very reliable, nor scalable, but it is good enough to keep Unix creaking along. | ||
| Line 655: | Line 664: | ||
| ====== Part 10: C++ ====== | ====== Part 10: C++ ====== | ||
| - | "The COBOL of the 90s" | + | <blockquote>The COBOL of the 90s</blockquote> |
| - | "Q. Where did the names 'C' and 'C++' come from? | + | <blockquote> |
| - | A. They were grades" -Jerry Leichter. | + | Q. Where did the names 'C' and 'C++' come from? |
| + | A. They were grades</blockquote> | ||
| It was perhaps inevitable that out of the Unix philosophy of not ever making anything easy for the user would come a language like C++. The idea of object-oriented programming dates back to Simula in the 60s, hitting the big time with Smalltalk in the early 70s. Other books can tell you how using any of dozen of object-oriented languages can make programmers more productive, make code more robust, and reduce maintenance costs. Don't except to see any of these advantages in C++. | It was perhaps inevitable that out of the Unix philosophy of not ever making anything easy for the user would come a language like C++. The idea of object-oriented programming dates back to Simula in the 60s, hitting the big time with Smalltalk in the early 70s. Other books can tell you how using any of dozen of object-oriented languages can make programmers more productive, make code more robust, and reduce maintenance costs. Don't except to see any of these advantages in C++. | ||
| Line 666: | Line 676: | ||
| Comparing C++ to COBOL is unfair to COBOL, which actually was a marvelous feat of engineering, given the technology of its day. The only marvelous thing about C++ is that anyone manages to get any work done in it at all. Fortunately, most good programmers know that they can avoid c++ by writing largely in C, steering clear of most of the ridiculous features that they'll probably never understand anyway. Usually, this means writing their own non-object-oriented tools to get just the features they need. Of course, this means their code will be idiosyncratic, incompatible, and impossible to understand or reuse. But a thin veneer of C++ here and there is just enough to fool managers into approving their projects. | Comparing C++ to COBOL is unfair to COBOL, which actually was a marvelous feat of engineering, given the technology of its day. The only marvelous thing about C++ is that anyone manages to get any work done in it at all. Fortunately, most good programmers know that they can avoid c++ by writing largely in C, steering clear of most of the ridiculous features that they'll probably never understand anyway. Usually, this means writing their own non-object-oriented tools to get just the features they need. Of course, this means their code will be idiosyncratic, incompatible, and impossible to understand or reuse. But a thin veneer of C++ here and there is just enough to fool managers into approving their projects. | ||
| - | ==== The assembly language of Object-oriented-programming. ==== | + | ===== The assembly language of Object-oriented-programming. ===== |
| There's nothing high-level about C++. To see why, let us look at properties of a true high-level language: | There's nothing high-level about C++. To see why, let us look at properties of a true high-level language: | ||
| Line 678: | Line 688: | ||
| A low-level language demands attention to myriad details, most of which have more to do with the machine's internal operation that with the problem being solved. Not only does this make the code inscrutable, but it builds in obsolescence. As new system come along, practically every other year these days, low-level code becomes out of date and must be manually patched or converted at enormous expense. | A low-level language demands attention to myriad details, most of which have more to do with the machine's internal operation that with the problem being solved. Not only does this make the code inscrutable, but it builds in obsolescence. As new system come along, practically every other year these days, low-level code becomes out of date and must be manually patched or converted at enormous expense. | ||
| - | ==== Pardon me, your memory is leaking... ==== | + | ===== Pardon me, your memory is leaking... ===== |
| It's well known that the vast majority of program errors have to do with memory mismanagement. Before you can use an object, you have to allocate some space for it, initialize it properly, keep track of it somehow, and dispose of it properly. Of course, each of these tasks is extraordinarily tedious and error-prone, with disastrous consequences for the slightest error. Detecting and correcting these mistakes are notoriously difficult, because they are often sensitive to subtle differences in configuration and usage patterns for different users. | It's well known that the vast majority of program errors have to do with memory mismanagement. Before you can use an object, you have to allocate some space for it, initialize it properly, keep track of it somehow, and dispose of it properly. Of course, each of these tasks is extraordinarily tedious and error-prone, with disastrous consequences for the slightest error. Detecting and correcting these mistakes are notoriously difficult, because they are often sensitive to subtle differences in configuration and usage patterns for different users. | ||
| Line 692: | Line 702: | ||
| C++ users, alas, are forced to pick up their garbage manually. Many have brainwashed into thinking that somehow this is more efficient that using something written by experts specially for the platform they use. | C++ users, alas, are forced to pick up their garbage manually. Many have brainwashed into thinking that somehow this is more efficient that using something written by experts specially for the platform they use. | ||
| - | === The evolution of a programmer. === | + | ===== The evolution of a programmer. ===== |
| /*High school/Junior high*/ | /*High school/Junior high*/ | ||
| Line 769: | Line 779: | ||
| ====== Part 11: System Administration ====== | ====== Part 11: System Administration ====== | ||
| - | "Unix's Hidden Cost" | + | <blockquote>Unix's Hidden Cost</blockquote> |
| - | "If the automobile had followed the same development as the computer, a Rolls-Royce | + | <blockquote>If the automobile had followed the same development as the computer, a Rolls-Royce |
| would today cost $100, get a million miles per gallon, and explode once a year | would today cost $100, get a million miles per gallon, and explode once a year | ||
| - | killing everyone inside." -Robert Cringely, InfoWorld. | + | killing everyone inside.</blockquote> |
| All Unix systems require a System Administrator, affectionately known as a Sysadmin. The sysadmin's duties include: | All Unix systems require a System Administrator, affectionately known as a Sysadmin. The sysadmin's duties include: | ||
| Line 790: | Line 800: | ||
| Paying someone $40,000 a year to maintain 20 machines translates into $2000 per machine-year. Typical low-end Unix workstation cost between $3000 and $5000 and are replaced about every two years. Combine these costs with the cost of the machine and software, it becomes clear that the allegedly cost-effective "solution" of "open systems" isn't really cost-effective at all. | Paying someone $40,000 a year to maintain 20 machines translates into $2000 per machine-year. Typical low-end Unix workstation cost between $3000 and $5000 and are replaced about every two years. Combine these costs with the cost of the machine and software, it becomes clear that the allegedly cost-effective "solution" of "open systems" isn't really cost-effective at all. | ||
| - | ==== Keeping Unix running and tuned. ==== | + | ===== Keeping Unix running and tuned. ===== |
| Sysadmins are highly paid baby sitters. Just as a baby transforms perfectly good input into excrement, which it then drops in its diapers. Unix drops excrement all over its file system and the network in the form of dumps from crashing programs, temporary files that aren't, cancerous log files, and illegitimate network rebroadcasts. But unlike the baby, who may smear his nuggets around but generally keeps them in his diapers. Unix play hide and seek with its waste. Without an experienced sysadmin to ferret them out, the system slowly runs out of space, starts to stink, gets uncomfortable, and complains or just dies. | Sysadmins are highly paid baby sitters. Just as a baby transforms perfectly good input into excrement, which it then drops in its diapers. Unix drops excrement all over its file system and the network in the form of dumps from crashing programs, temporary files that aren't, cancerous log files, and illegitimate network rebroadcasts. But unlike the baby, who may smear his nuggets around but generally keeps them in his diapers. Unix play hide and seek with its waste. Without an experienced sysadmin to ferret them out, the system slowly runs out of space, starts to stink, gets uncomfortable, and complains or just dies. | ||
| Line 800: | Line 810: | ||
| It's not surprising that most major Unix systems suffer from memory leaks, garbage accumulation, and slow corruption of their address space -problems that typically only show themselves after a program has been running for a few days. | It's not surprising that most major Unix systems suffer from memory leaks, garbage accumulation, and slow corruption of their address space -problems that typically only show themselves after a program has been running for a few days. | ||
| - | ==== Disk partitions and backups. ==== | + | ===== Disk partitions and backups. ===== |
| Disk space management is a chore on all types of computer systems; on Unix, it's a Herculean task. Before loading Unix onto your disk, you must decide upon a space allocation for each of Unix's partitions. Unix pretends your disk drive as a collection of smaller disks (each containing a complete file system), as opposed to other systems like TOP-20, which let you create a larger logical disk out of a collection of smaller physical disks. | Disk space management is a chore on all types of computer systems; on Unix, it's a Herculean task. Before loading Unix onto your disk, you must decide upon a space allocation for each of Unix's partitions. Unix pretends your disk drive as a collection of smaller disks (each containing a complete file system), as opposed to other systems like TOP-20, which let you create a larger logical disk out of a collection of smaller physical disks. | ||
| Line 810: | Line 820: | ||
| The problem of fixed size disk partitions still hurts less now that gigabyte disks are standard equipment. The manufacturers ship machines with disk partition large enough to avoid problems. It's a relatively expensive solution, but much easier to implement that fixing Unix. | The problem of fixed size disk partitions still hurts less now that gigabyte disks are standard equipment. The manufacturers ship machines with disk partition large enough to avoid problems. It's a relatively expensive solution, but much easier to implement that fixing Unix. | ||
| - | ==== Partitions: Twice the Fun. ==== | + | ===== Partitions: Twice the Fun. ===== |
| Because of Unix's tendency to trash its own file system, early Unix gurus developed a workaround to keep of their files from getting regularly trashed: partition the disk into separate spaces. If the system crashes, and you get lucky, only half your data will be gone. The file system gets trashed because the free list on disk is usually inconsistent. When Unix crashes, the disks with the most activity get the most corrupted, because those are the most inconsistent disks -that is, they had the greatest amount of information in memory and not on the disk. The gurus decided to partition the disks instead, dividing a simple physical disk into several, smaller, virtual disks, each with its own file system. | Because of Unix's tendency to trash its own file system, early Unix gurus developed a workaround to keep of their files from getting regularly trashed: partition the disk into separate spaces. If the system crashes, and you get lucky, only half your data will be gone. The file system gets trashed because the free list on disk is usually inconsistent. When Unix crashes, the disks with the most activity get the most corrupted, because those are the most inconsistent disks -that is, they had the greatest amount of information in memory and not on the disk. The gurus decided to partition the disks instead, dividing a simple physical disk into several, smaller, virtual disks, each with its own file system. | ||
| Line 821: | Line 831: | ||
| ====== Part 12: Security ====== | ====== Part 12: Security ====== | ||
| - | "Oh, I'm sorry, Sir, Go ahead, I didn't realize you were root" | + | <blockquote>Oh, I'm sorry, Sir, Go ahead, I didn't realize you were root</blockquote> |
| - | "Unix is computer-scientology, not computer science." -Dave Mankins. | + | <blockquote>Unix is computer-scientology, not computer science.</blockquote> |
| The term "Unix security" is, almost by definition, an oxymoron because the Unix operating system was not designed to be secure, except for the vulnerable and ill-designed root/rootless distinction. Security measures to thwart attack were an afterthough. Thus, when Unix is behaving as expected, it is not secure, and making Unix run "securely" means forcing it to do unnatural acts. It's like the dancing dog at a circus, but not as funny -especially when it is your files that are being eaten by the dog. | The term "Unix security" is, almost by definition, an oxymoron because the Unix operating system was not designed to be secure, except for the vulnerable and ill-designed root/rootless distinction. Security measures to thwart attack were an afterthough. Thus, when Unix is behaving as expected, it is not secure, and making Unix run "securely" means forcing it to do unnatural acts. It's like the dancing dog at a circus, but not as funny -especially when it is your files that are being eaten by the dog. | ||
| - | ==== Holes in the armor. ==== | + | ===== Holes in the armor. ===== |
| Two fundamental design flaws prevent Unix from being secure. First, Unix stores security information about the computer inside the computer itself, without encryption or the other mathematical protections. It's like leaving the keys to your safe sitting on your desk: as soon as an attacker breaks through the Unix front door, he's compromised the entire system. Second, the Unix superuser concept is a fundamental security weakness. | Two fundamental design flaws prevent Unix from being secure. First, Unix stores security information about the computer inside the computer itself, without encryption or the other mathematical protections. It's like leaving the keys to your safe sitting on your desk: as soon as an attacker breaks through the Unix front door, he's compromised the entire system. Second, the Unix superuser concept is a fundamental security weakness. | ||
| - | Superuser:The Superflaw | + | ==== Superuser:The Superflaw ==== |
| All multiuser operating systems need privileged accounts. Virtually all multiuser operating systems other than Unix apportion privilege according to need. Unix's "superuser" is all-or-nothing. An administrator who can change people's passwords must also, by design, be able to wipe out every file on the system. That high school kid you've hired to do backups might accidentally (or intentionally) leave your system open to attack. | All multiuser operating systems need privileged accounts. Virtually all multiuser operating systems other than Unix apportion privilege according to need. Unix's "superuser" is all-or-nothing. An administrator who can change people's passwords must also, by design, be able to wipe out every file on the system. That high school kid you've hired to do backups might accidentally (or intentionally) leave your system open to attack. | ||
| Line 849: | Line 859: | ||
| When combined with removable media (such as floppy disks or SyQuest drives), SUID gives the attacker a power full way to break into otherwise "secure" systems: simply put a SUID root file on a floppy disk and mount it, the run the SUID root program to become root. (The Unix-savvy reader might object to this attack, saying that mount is a privileged command that requires superuser privileges to run. Unfortunately, many manufacturers now provide SUID programs for mounting removable media specifically to ameliorate this "inconvenience".) | When combined with removable media (such as floppy disks or SyQuest drives), SUID gives the attacker a power full way to break into otherwise "secure" systems: simply put a SUID root file on a floppy disk and mount it, the run the SUID root program to become root. (The Unix-savvy reader might object to this attack, saying that mount is a privileged command that requires superuser privileges to run. Unfortunately, many manufacturers now provide SUID programs for mounting removable media specifically to ameliorate this "inconvenience".) | ||
| - | ==== Processes are cheap -and dangerous. ==== | + | ===== Processes are cheap -and dangerous. ===== |
| Another software tool for breaking Unix security are the systems calls fork() and exec(), which enable one program to spawn other programs. Programs spawning subprograms lie at the heart of Unix's tool-based philosophy. Emacs and FTP run sub processes to accomplish specific tasks such as listing files. The problem for the security-conscious is that these programs inherit the privileges of the programs that spawn them. | Another software tool for breaking Unix security are the systems calls fork() and exec(), which enable one program to spawn other programs. Programs spawning subprograms lie at the heart of Unix's tool-based philosophy. Emacs and FTP run sub processes to accomplish specific tasks such as listing files. The problem for the security-conscious is that these programs inherit the privileges of the programs that spawn them. | ||
| Line 855: | Line 865: | ||
| Easily spawned sub processes are a two-edged sword because a spawned subprogram can be a shell that lowers the drawbridge to let the Mongol hordes in. When the spawning program is running as superuser, then its spawned process also run as superuser. Many a cracker has gained entry through spawned superuser shells. | Easily spawned sub processes are a two-edged sword because a spawned subprogram can be a shell that lowers the drawbridge to let the Mongol hordes in. When the spawning program is running as superuser, then its spawned process also run as superuser. Many a cracker has gained entry through spawned superuser shells. | ||
| - | ==== The problem with PATH ==== | + | ===== The problem with PATH ===== |
| Unix has to locate the executable image that corresponds to a given command name. To find the executable, Unix consults the user's PATH variable for a list of directories to search. For example, if your PATH environment is :/bin:/usr/bin:/etc:/usr/local/bin then, when you type snarf, Unix will automatically search through the /bin, /usr/bin/, /etc, and /usr/local/bin directories, in that order, for a program snarf. | Unix has to locate the executable image that corresponds to a given command name. To find the executable, Unix consults the user's PATH variable for a list of directories to search. For example, if your PATH environment is :/bin:/usr/bin:/etc:/usr/local/bin then, when you type snarf, Unix will automatically search through the /bin, /usr/bin/, /etc, and /usr/local/bin directories, in that order, for a program snarf. | ||
| Line 863: | Line 873: | ||
| PATH=:.:/bin:/usr/bin:/usr/local/bin: | PATH=:.:/bin:/usr/bin:/usr/local/bin: | ||
| - | ==== Startup traps ==== | + | ===== Startup traps ===== |
| When a complicated Unix program starts up, it reads configuration files from either the user's home directory and/or the current directory to set initial and default parameters that customize the program to the user's specifications. Unfortunately, start up files can be created and left by other users to do their bidding on your behalf. | When a complicated Unix program starts up, it reads configuration files from either the user's home directory and/or the current directory to set initial and default parameters that customize the program to the user's specifications. Unfortunately, start up files can be created and left by other users to do their bidding on your behalf. | ||
| Line 876: | Line 886: | ||
| ====== Part 13: The File System ====== | ====== Part 13: The File System ====== | ||
| + | <blockquote> | ||
| + | Sure, it corrupts your files, but look how fast it is!</blockquote> | ||
| - | "Sure, it corrupts your files, but look how fast it is!" | + | <blockquote>Pretty daring of you to be storing important files on a Unix system.</blockquote> |
| - | + | ||
| - | "Pretty daring of you to be storing important files on a Unix system." -Robert E. Seastrom. | + | |
| The traditional Unix file system is a grotesque hack that, over the years, has been enshrined as a "standard" by virtue of its widespread use. Indeed after years of indoctrination and brainwashing, people now accept Unix's flaws as desired features. It's like a cancer victim's immune system enshrining the carcinom a cell as ideal because the body is so good at making them. | The traditional Unix file system is a grotesque hack that, over the years, has been enshrined as a "standard" by virtue of its widespread use. Indeed after years of indoctrination and brainwashing, people now accept Unix's flaws as desired features. It's like a cancer victim's immune system enshrining the carcinom a cell as ideal because the body is so good at making them. | ||
| Line 887: | Line 897: | ||
| But the real faults of Unix file systems run far deeper that these two missing features. The faults are not faults of execution, but of ideology. With Unix, we often are told that "everything is a file". Thus, it's not surprising that many of Unix's fundamental faults lie with the file system as well. | But the real faults of Unix file systems run far deeper that these two missing features. The faults are not faults of execution, but of ideology. With Unix, we often are told that "everything is a file". Thus, it's not surprising that many of Unix's fundamental faults lie with the file system as well. | ||
| - | ==== What's a file system? ==== | + | ===== What's a file system? ===== |
| A file system is the part of a computer's operating system that manages file storage on mass-storage devices such as floppy disks and hard drives. Each piece of information has a name, called the filename, and a unique place (we hope) on the hard disk. The file system's duty is to translate names such as /etc/passwd into locations on the disk such as "block 32156 of hard disk #2". It also supports the reading and writing of a file's blocks. Although conceptually a separable part of the operating system, in practice, nearly every operating system in use today comes with its own peculiar file system. | A file system is the part of a computer's operating system that manages file storage on mass-storage devices such as floppy disks and hard drives. Each piece of information has a name, called the filename, and a unique place (we hope) on the hard disk. The file system's duty is to translate names such as /etc/passwd into locations on the disk such as "block 32156 of hard disk #2". It also supports the reading and writing of a file's blocks. Although conceptually a separable part of the operating system, in practice, nearly every operating system in use today comes with its own peculiar file system. | ||
| - | ==== Meet the relatives. ==== | + | ===== Meet the relatives. ===== |
| In the past two decades, the evil stepmother Unix has spawned not one, not two, but four different file systems. These step-systems all behave slightly differently when running the same program under the same circumstances. | In the past two decades, the evil stepmother Unix has spawned not one, not two, but four different file systems. These step-systems all behave slightly differently when running the same program under the same circumstances. | ||
| Line 903: | Line 913: | ||
| Sun began the Network File System NFS. NFS allegedly lets different networked Unix computers share files "transparently". With NFS, one computer is designated as a "file server", and another computer is called the "client". The (somewhat dubious) goal is for the files and file hierarchies on the server to appear more or less on the client in more or less the same way that they appear on the server. Although Apollo Computers had a network file system that worked better than NFS several years before NFS was a commercial product, NFS became the dominant standard because it was "operating system independent" and Sun promoted it as an "open standard". | Sun began the Network File System NFS. NFS allegedly lets different networked Unix computers share files "transparently". With NFS, one computer is designated as a "file server", and another computer is called the "client". The (somewhat dubious) goal is for the files and file hierarchies on the server to appear more or less on the client in more or less the same way that they appear on the server. Although Apollo Computers had a network file system that worked better than NFS several years before NFS was a commercial product, NFS became the dominant standard because it was "operating system independent" and Sun promoted it as an "open standard". | ||
| - | ==== Visualize a File System. ==== | + | ===== Visualize a File System. ===== |
| Take a few moments to imagine what features a good file system might provide to an operating system, and you'll quickly see the problems shared by all of the file systems described in this chapter. | Take a few moments to imagine what features a good file system might provide to an operating system, and you'll quickly see the problems shared by all of the file systems described in this chapter. | ||
| Line 913: | Line 923: | ||
| Advanced file systems exploit the features of modern hard disk drives and controllers. For example, since most disk drives can Transfer up to 64 Kbytes in a single burst, advanced file systems store files in contiguous blocks so they can be read and written in a single operation. They also have support for scatter/gather operations, so many individual reads or writes can be batched up and executed as one. | Advanced file systems exploit the features of modern hard disk drives and controllers. For example, since most disk drives can Transfer up to 64 Kbytes in a single burst, advanced file systems store files in contiguous blocks so they can be read and written in a single operation. They also have support for scatter/gather operations, so many individual reads or writes can be batched up and executed as one. | ||
| - | ==== No File Types ==== | + | ===== No File Types ===== |
| To UFS and all Unix-derived file systems, files are nothing more than long sequences of bytes. (A bag'o' bytes, as the mythology goes, even though they are technically not bags, but streams). Programs are free to interpret those bytes however they wish. To make this easier, Unix doesn't store type information with each file. Instead, Unix forces the user to encode this information in the file's name! Files ending with a ".c" are C source files, files ending with a ".o" are object files, and so forth. This makes it easy to burn your fingers when renaming files. | To UFS and all Unix-derived file systems, files are nothing more than long sequences of bytes. (A bag'o' bytes, as the mythology goes, even though they are technically not bags, but streams). Programs are free to interpret those bytes however they wish. To make this easier, Unix doesn't store type information with each file. Instead, Unix forces the user to encode this information in the file's name! Files ending with a ".c" are C source files, files ending with a ".o" are object files, and so forth. This makes it easy to burn your fingers when renaming files. | ||
| Line 919: | Line 929: | ||
| To resolve this problem, some Unix files have "magic numbers" that are contained in the file's first few bytes. Only some files -shell scripts, ".o" files and executable programs -have magic numbers. What happens when a file's "type" (as indicated by its extension) and its magic number don't agree? That depends on the particular program you happen to be running. The loader will just complain and exit. The exec() family of kernel functions, on the other hand, might try starting up a copy of /bin/sh and giving your file to that shell as input. | To resolve this problem, some Unix files have "magic numbers" that are contained in the file's first few bytes. Only some files -shell scripts, ".o" files and executable programs -have magic numbers. What happens when a file's "type" (as indicated by its extension) and its magic number don't agree? That depends on the particular program you happen to be running. The loader will just complain and exit. The exec() family of kernel functions, on the other hand, might try starting up a copy of /bin/sh and giving your file to that shell as input. | ||
| - | ==== Only the Most Perfect Disk Pack Need Apply ==== | + | ===== Only the Most Perfect Disk Pack Need Apply ===== |
| One common problem with Unix is perfection: while offering none of its own, the operating system demands perfection from the hardware upon which it runs. That's because Unix programs usually don't check for hardware error -they just blindly stumble along when things begin to fail, until they trip and panic. | One common problem with Unix is perfection: while offering none of its own, the operating system demands perfection from the hardware upon which it runs. That's because Unix programs usually don't check for hardware error -they just blindly stumble along when things begin to fail, until they trip and panic. | ||
| Line 927: | Line 937: | ||
| ====== Part 14: NFS ====== | ====== Part 14: NFS ====== | ||
| - | "Nightmare file system" | + | <blockquote>Nightmare file system</blockquote> |
| - | "The "N" in NFS stands for Not, or Need, or perhaps Nightmare." -Henry Spencer. | + | <blockquote> |
| + | The "N" in NFS stands for Not, or Need, or perhaps Nightmare.</blockquote> | ||
| In the mid-1980s, Sun Microsystems developed a system for letting computers share files over a network. Called the Network file system -or more often NFS- this system was largely responsible for Sun's success as a computer manufacturer. NFS let Sun sell bargain-basement "diskless" workstations that stored files on larger "file servers." all made possible through the of Xerox's Ethernet technology. | In the mid-1980s, Sun Microsystems developed a system for letting computers share files over a network. Called the Network file system -or more often NFS- this system was largely responsible for Sun's success as a computer manufacturer. NFS let Sun sell bargain-basement "diskless" workstations that stored files on larger "file servers." all made possible through the of Xerox's Ethernet technology. | ||
| - | ==== Not fully serviceable. ==== | + | ===== Not fully serviceable. ===== |
| NFS is based on the concept of the "magic cookie". Every file and every directory on the file server is represented by a magic cookie. To read a file, you send the file server a packet containing the file's magic cookie and the range of bytes that you want to read. The file server sends you back a packet with the bytes. Likewise, to read the contents of a directory, you send the server the directory's magic cookie. The server sends you back a list of the files that are in the remote directory, as well as a magic cookie for each of the files that the remote directory contains. | NFS is based on the concept of the "magic cookie". Every file and every directory on the file server is represented by a magic cookie. To read a file, you send the file server a packet containing the file's magic cookie and the range of bytes that you want to read. The file server sends you back a packet with the bytes. Likewise, to read the contents of a directory, you send the server the directory's magic cookie. The server sends you back a list of the files that are in the remote directory, as well as a magic cookie for each of the files that the remote directory contains. | ||
| Line 943: | Line 954: | ||
| "Stateless" means that all of the information that the client needs to mount a remote file system is kept on the client, instead of having additional information stored on the server. Once a magic cookie is issued for a file, that file handle will remain good even if the server is shut down and rebooted, as long as the file continues to exist and no major changes are made to the configuration of the server. | "Stateless" means that all of the information that the client needs to mount a remote file system is kept on the client, instead of having additional information stored on the server. Once a magic cookie is issued for a file, that file handle will remain good even if the server is shut down and rebooted, as long as the file continues to exist and no major changes are made to the configuration of the server. | ||
| - | ==== Broken cookie. ==== | + | ===== Broken cookie. ===== |
| Over the years, Sun has discovered many cases in which the NFS breaks down. Rather than fundamentally redesign NFS, all Sun has done is hacked upon it. | Over the years, Sun has discovered many cases in which the NFS breaks down. Rather than fundamentally redesign NFS, all Sun has done is hacked upon it. | ||
| Line 967: | Line 978: | ||
| Why the hack doesn't work: If the client crashes, the dot-file never gets deleted. As a result, NFS servers have to run nightly "clean-up" shell scripts that search for all of the files with names like ".nfs0003234320" that are more than a few days old and automatically delete them. This is why most Unix systems suddenly freeze up at 2:00 a.m. each morning -they're spinning their disks running find. And you better not go on vacation with the mail(1) program still running if you want your mail file to be around when you return. (No kidding!) | Why the hack doesn't work: If the client crashes, the dot-file never gets deleted. As a result, NFS servers have to run nightly "clean-up" shell scripts that search for all of the files with names like ".nfs0003234320" that are more than a few days old and automatically delete them. This is why most Unix systems suddenly freeze up at 2:00 a.m. each morning -they're spinning their disks running find. And you better not go on vacation with the mail(1) program still running if you want your mail file to be around when you return. (No kidding!) | ||
| - | ==== Virtual file corruption. ==== | + | ===== Virtual file corruption. ===== |
| What's better than a networked file system that corrupts your files? A file system that doesn't really corrupt them, but only makes them appear as if they are corrupted. NFS does this from time to time. One of the reason that NFS silently corrupts files is that, by default, NFS is delivered with UDP checksum error-detection systems turned off. Makes sense, doesn't it? After all, calculating checksums takes a long time, and the net is usually reliable. At least, that was the state-of-the-art back in 1984 and 1985, when these decisions were made. | What's better than a networked file system that corrupts your files? A file system that doesn't really corrupt them, but only makes them appear as if they are corrupted. NFS does this from time to time. One of the reason that NFS silently corrupts files is that, by default, NFS is delivered with UDP checksum error-detection systems turned off. Makes sense, doesn't it? After all, calculating checksums takes a long time, and the net is usually reliable. At least, that was the state-of-the-art back in 1984 and 1985, when these decisions were made. | ||