Mark Minasi's Windows Networking Tech Page
Issue #33 April 2003

To subscribe, visit http://www.minasi.com/nwsreg.htm. To unsubscribe, link to http://www.minasi.com/unsubs.htm. To change e-mail address, switch between HTML or text format, etc., link to http://www.minasi.com/edit-newsletter-record.htm.  Visit the Archives at http://www.minasi.com/archive.htm.  Please do NOT reply to this mail; for comments, please link to www.minasi.com/gethelp.  Document copyright 2003 Mark Minasi.

What's Inside

  • News: 
    • Free Download: Mastering Windows Server 2003, Chapter 1
    • Buy Mastering Windows Server 2003 and Getting Ready for .NET Server 2003 at 55% Off
    • Upcoming Free Appearances: NetIQ Web Cast 7 May, Windows Decisions 14/15 May
    • Seminars: XP and the NEW Server 2003/2000 Classes: LA, DC, NY
    • The Newsletter Database Conversion is Done
    • New Member Option:  URL-Only
    • Join Me In Denmark May 28/29!
  • Tech Section
    • Essential Security Tip:  Get Rid of NTLM and LM Authentication
    • Linux Book Errata
  • Conferences
  • Bring a Seminar to Your Site

News

Hello all —

It's finally here; I just got my copies of Mastering Windows Server 2003.  And my publisher, Sybex, has allowed me to give some of it away free.  Also in this month, I explain how to plug a pretty scary security hole that still exists in all versions of Windows — LM authentication.  Believe it or not, 99.9% of the systems that I see on client sites still keep this incredibly easy-to-crack password system running in tandem with their normal NT 4 or Active Directory domain passwords, even though fixing it is pretty easy.  I've also got a pretty comprehensive list of errata for the Linux for Windows Administrators book in this issue.

Well, I didn't mean for this to turn out this way, but this is my largest newsletter ever.  So it's fitting that I can offer a large deal in this issue — the chance to buy the new Server book and the 2003 overview CD for 55% off.  But I've got some other great news too; I've reworked the newsletter subscriber database for far faster response time, and you can now register to only receive an URL pointing to the newsletter rather than getting the whole newsletter in your in-box, as many of you have requested.

Free Download: Mastering Windows Server 2003, Chapter 1

After six months of almost nonstop writing, my book on Windows Server 2003 is finally here.  As I write this, readmedoc.com's Web site says that they're shipping; Amazon and Bookpool's sites say that they don't have the book yet although I'm pretty sure they do — I'm guessing they haven't updated their Web pages on the book.

There's a lot to tell you about the book and I didn't want to end up devoting an entire newsletter to it, so I've got a page with information about it at www.minasi.com/ws2003book.htm.  There's also a link to download the first chapter from the book; I encourage you to pull it down, it's just 20 pages and offers an overview of the new stuff in 2003.

Buy Mastering Windows Server 2003 and the Getting Ready for 2003 CD for 55% Off

I've been trying to work out some kind of deal to offer books at a good price to my newsletter subscribers.  My friends at Xanadu Books have put together an offer that I hope you'll like.  You may know that I've been offering my 75 minute audio CD overview of Windows Server 2003, "Getting Ready for .NET Server" — they changed the name just before they released the product, but nothing else, hence the .NET name in the title — for $29.95, and we've sold tons of them at that price.  The new Server 2003 book lists for $59.95.  Xanadu's selling the two bundled, an $89.90 value, for US$39.95 plus shipping.  Visit http://www.minasi.com/2003booksale to find out more. 

Upcoming Free Appearances: NetIQ Webcast 7 May, Windows Decisions 14/15 May

This seems to be the month for free stuff.  NetIQ is doing a webcast on making Active Directory work; sign up at http://www.netiq.com/events/webcasts/activedirectory/default.asp and mark your calendars for noon Eastern time on 7 May.  And don't forget that the SearchWin2000.com people are organizing a conference featuring some of your favorite techie geek speakers (including Roberta Bragg, Jeremy Moskowitz, my old co-worker at Windows NT Magazine, John Enck and a bunch of others).  Best of all, the conference is free.  In addition to talks like my "State of The OS" talk, where you can learn how buffer overrun errors are very much like international relations, there's also a "meet the experts" where you can come ask Roberta how to solve your security problems.  It's in Chicago May 14-16, sign up at http://windowsdecisions.techtarget.com/html/registration.htm?Offer=wdmkmn1.  Budgets are tight; free is good...

Seminars: XP and the NEW Server 2003/2000 Classes: LA, DC, NY

... but of course not everything can be free.  If it can't, however, it can be cost-effective!  That's what we aim for in our popular two-day desktop and server support seminars.  Our next XP and 2003/2000 Active Directory seminars are next coming to Los Angeles in mid-July, then Washington DC in September and New York City in November.  Find out about the XP seminar at http://www.minasi.com/xpsupport.htm,  the Active Directory/Group Policy seminar at http://www.minasi.com/2003outln.htm, and the schedule of seminars at http://www.minasi.com/pubsems.htm

The Newsletter Database Conversion is Done

If you've ever retrieved a password or updated your e-mail address in the newsletter subscriber database, or accessed an archive copy of the newsletter, then you might have had to wait for a minute or two to log in.  That was because I previously kept the database on a simple ASCII text file.  (Not optimal, I know, but it was quick to code and, umm, that's why I don't write programming books.)

I've been working to re-code the newsletter subscriber database so that it runs off a SQL server and I'm happy to announce that it's finally done.  Here are some of the links affected (they're on the home page as well):

Two New Audio CDs: "12 Tips to Secure Your Network" and "Tuning 2000, XP and 2003"

As promised, we've developed and are shipping two new audio CDs.  We're now selling a two-CD talk "12 Tips to Secure Your Network," an expanded version of a talk that thousands have attended, and a one-CD talk "Tuning 2000, XP and 2003," the lowdown on getting the most out of your systems.  Info on the security talk at www.minasi.com/secaudio.htm and info on the tuning talk at www.minasi.com/tuneaudio.htm

New Member Option:  URL-Only

When you signed up for this newsletter, I offered two delivery options — I'd e-mail you the newsletter as text, or as HTML.  Now there's a third option: URL-only.  If you choose that, then you'll just get a very short text e-mail telling you that the newsletter's been posted to a Web site and the URL to view it.  I offer this for two reasons:  first, some people have asked for it, and, second, some of you just plain don't get this newsletter because your company's e-mail servers have hyperactive spam filters.  (Some people tell me that just having the word "newsletter" in the subject line causes the e-mail to be deleted!)  To change your subscription option, access the membership editing page at this URL:

http://www.minasi.com/edit-newsletter-record.htm

I hope this is a useful new option for many of you.  No matter what delivery system you've selected, I still send a short text e-mail announcing the newsletter so that those with ravenous spam filters can expect the latest issue.

Join Me In Denmark May 28/29!

My European friends often e-mail to ask when I'll be speaking on The Continent.  Now I've got an answer — April 29/30 in Denmark.  The IT Solutions Group has hired me to present several topics over two days, including an overview of what's new in Windows Server 2003, some in-depth discussion of AD maintenance, troubleshooting and migration, how to secure a network, and how to tune systems.  It'll be fast-paced, educational and fun, so I hope you'll come join me!  More info at http://TheITsolutionsGroup.com/events.htm.

Tech Section

The Final Set of XP/2003-Friendly Admin Tools Are Released

After what seemed like weekly updates, the RTM (release to manufacturing) version of the administrative tools for XP and 2003 are now available at http://microsoft.com/downloads/details.aspx?familyid=c16ae515-c8f4-47ef-a1e4-a8dcbacff8e3&displaylang=en and sadly they're as irritating to install as the old ones were —  make sure all of the old versions are removed, perhaps fiddle with the Registry and so on.  Double sadly, they no longer work on Windows 2000 systems and I've had some trouble getting some XP systems to install it correctly, even with SP1 and the old tools removed.

Featured Article: Stomping the LM Authentication Hole

Anyone reading Microsoft security PR stuff would be led to believe that running a modern NT/2000/XP/2003 system would reap the benefits of their newer, more secure operating system software.  And that's true in the sense that their newest authentication systems, things called NTLMv2 and Kerberos, are indeed quite good.  But there's an old security hole that's existed in NT since its earliest days that is easily exploited (which is bad) but easily closed for free (which is good).  In other words, having a state-of-the-art Active Directory or even an NT 4-based domain can be very secure but for one thing:  there's some software hanging around from the late 1980s that every single Microsoft system runs just on the off-chance that you're still running a LAN Manager 2.1 server, and that software (called "LM authentication") can be used to steal passwords on your system.  In this article, I want to explain basically how Microsoft systems do authentications, how this older software creates a security hole, and how to close that hole.  I also want to advocate strongly getting rid of something called "NTLM" in favor of "NTLMv2."  So I've got  a lot of explaining to do, but trust me, it'll be worthwhile.

Over the years, Microsoft has created a variety of protocols designed to allow people to authenticate to network servers securely.  First there was LM, the authentication system used by LAN Manager, an OS/2-based network operating system available in the 1989-1994 time frame.  Windows for Workgroups, Windows 9x and ME systems still use LM.  Then 1993 saw an improvement in LM with the introduction of NT 3.1 through a new authentication system called NTLM.  By default all NT, 2000, XP, and 2003 systems use NTLM to log onto workgroup members or NT 4 domains.  NT 4 systems also use NTLM to log onto Active Directory domains.  NT 4.0 Service Pack 4 introduced more improvements to NTLM, resulting in NTLMv2.  I don't know of any operating systems that use NTLMv2 by default but, as you'll see, I hope you'll configure your systems to use it.  Finally, 2000, XP and 2003 systems can log onto Active Directory domains using Kerberos, which is an even more secure system than NTLMv2.  There is currently no way to make a Windows 9x, ME, or NT 4 system log onto an Active Directory using Kerberos.  In short, then, we've got four authentication systems to work with — LM, NTLM, NTLMv2, and Kerberos.  I'll talk about the first three in this article.

Challenges: Sending Passwords Without Transmitting Them 

No matter how much talk we hear about using fingerprints, retinas, smartcards or other high-tech methods to control access to network resources, 99.9 percent of us still control access to our servers with the same old simple approach:  passwords.  You claim to be someone, and prove it by showing that you know something that only that person knows.  Basically, then, every time you try to access a server, the server needs to verify that you are indeed who you say you are.  In the Microsoft world (and others also, but we're talking the Windows and NT families here) the server "challenges" your client (probably a workstation), in effect saying to the client "tell me the name and password of the person sitting at you so I can decide whether or not to grant the access that you want."

But how to exchange this password information?  At first glance, the simplest way to do this would be for the client to simply say something like "I'm working on behalf of a guy named Mark whose password is 'swordfish.'"  But you probably know that'd be a bad idea, as anyone on a network can easily run a "sniffer" that shows every byte transferred across the network.  If the server simply asked the workstation, "please send me your user's password," then anyone could sniff the network and see that information.  So, instead, servers and clients check passwords by sending data back and forth to prove that they use the same password, but without actually sending the password.  Let's take a minute and see how that works, and how it might be compromised.

Vastly oversimplified, here's how a challenge authentication works.  The client says, "please give me access for Mark."  The server says "hmmm, Mark's password is 'swordfish.'  But I can't ask the workstation to send 'swordfish' over the wire.  So I'll convert 'swordfish' to a number."  This is simple to do, as you may recall that computers actually store everything as numbers — strings of ones and zeros — anyway.  Let's say that 'swordfish' works out to equal 143 in numeric form.  Again, it wouldn't — it'd be much larger — but it's easier to follow the example with simple numbers. The server then asks the client, "take Mark's password and divide it by 7, then send me only the remainder."

The workstation then says to itself, "well, I know that the password's 'swordfish,' and if I convert that to a number, I get 143.  Divide 143 by 7 and you get 20 with 3 left over."  So it sends the server the answer "3."  Of course, the server can do its own calculation and gets the same number.  The fact that the server computes the same remainder as the client leads the server to believe that the client has the same password for Mark's account as the server, and the challenge has been met, so to speak.  When that happens, the server makes a little note to itself that says something along the lines of "I believe that I'm talking to Mark through such-and-such client," so that the client needn't re-authenticate between each byte transmitted from server to client.  That little note to itself is called an "access token."

Attacking a Challenge-Based Authentication

So far, so good; we've proven that the client has the same password as the server for Mark's account.  Again, this was a vastly oversimplified explanation of a challenge, but I promise I'll fill in the blanks as we go.  Let's next see how some scum-sucking dirtbag might try to steal a password even if we used a challenge.  He'd put a sniffer on your network and see this:

Client:  Let me in.

Server: 7.

Client: 3.

Server: Okay, you're in.

Is that enough information for Joe Dirtbag to figure out your password?  It might be, or it might not be.  

Strengthening Challenge-Based Authentication With Random Challenges

First of all, what if the server always responded "7" when issuing a challenge?  That'd be troublesome and very helpful to the dirtbag.  In that case, all Joe Dirtbag has to do is to just sniff a logon and record the messages that Mark's client sends the server -- the "Let me in" and "3" responses in my simple example.  If we can assume that the server's challenge will always be "7," then our recorded answer of "3" will always cause a successful login.

Such an attack is called a "replay attack."  Notice that Joey D still doesn't know Mark's password.  But he does know how to log on as Mark, which is just as good.

He could even take it further and try to figure out Mark's password in a "server always challenges with '7'" world.  He could change his password — we're assuming that he's got some kind of account on the system if he's close enough to sniff the network — to some new value and then try a logon, sniffing his own logon.  If he ends up with a response whose value equals 3, then he knows that whatever he set his own password to equals Mark's password.  You could imagine Joe writing a program that loops through a list of every word in the dictionary, first changing Joe's password to that word, logging Joe on and sniffing the response to the '7' challenge and storing it.  He'd then have a nice handy-dandy way to reverse-engineer any response to a challenge into the password that led to that response, thereby knowing the user's password.

There are two defenses built into Microsoft's authentication protocols that avoid this.  First, you can set up domains, whether NT 4 or Active Directory, to only keep users from changing their passwords more than once a day.  Second, Microsoft authentication algorithms have always issued random challenge values.

(Time for a definition.  What I've been calling a "challenge value" — that is, the "7" — is called the nonce, the challenge, or the challenge token by different sources. )

How is that possible?  Well, first of all, nonces are pretty big, although it's not clear how big.  Some sources say that NTLM employs 64-bit nonces; others say 128-bit.  (I apologize for being fuzzy on this and other details about NTLM authentication, but apparently Microsoft hasn't completely documented the process and so different sources quote different values.)  A 64-bit nonce can have any one of 18,446,744,073,709,551,616 (that's over 18 sextillion) values, a 128-bit nonce can have any one of about 340,000,000,000,000,000,000,000,000,000,000,000,000 values (and please don't ask me what the word for that number would be, I haven't a clue). 

Microsoft even offers a third protection via NTLMv2, which issues nonces that are not only random, they're unique; that is, once a server sends a nonce with a given value, it'll never send a nonce with that value again.  Replay attacks are, then, completely impossible in NTLMv2.

Strengthening Challenge-Based Authentication With Better Hash Algorithms

So we've seen that random, non-repeating nonces make the chances of pre-cracking passwords based on pre-computed responses to particular nonces and replay attacks pretty bleak.   Assuming constantly-changing nonces, how could Joe Dirtbag next try to crack our passwords?

Recall my example where the server says just "7" and the client responds "3."  That's all that we'd see if sniffing the communication -- "Let me in," followed by "7," followed by "3," followed by some kind of welcome message.  Notice that the server never actually says what to do with the 7; the algorithm that says "take the password, make it a number, and divide it by the number that I send you and send me back the remainder" is built into the operating system — actually in a program called lsass.exe.  It would seem, then, that not telling the world what that client and server are actually doing with the two number, not explaining the division and remainder stuff, is something of a defense against hackers.  But is it a good defense?

In the long run, probably not.  Security types call the approach of not revealing the methods whereby you secure a network "security by obscurity."  To an extent, we all do security by obscurity in the sense that no companies that I know of post maps explaining exactly how they secure their networks.  Making the bad guys figure out what kind of firewall you've got, what operating system you run and the like at least slows them down.  But it's not great security in the authentication area for two reasons.  First, eventually someone just disassembles the code, figures out the algorithm and tells the world. And, second, the problem with using a secret challenge algorithm is that because only a small number of people know about it, then there might be a flaw in the algorithm that might make reversing it — "backwards-computing" the password from a challenge and a response — really easy, even if it wasn't obvious to the small number of people who knew the algorithm.  A better approach would be for a whole lot of smart people to figure out algorithms that are really hard to "backwards compute."

Before I go any further, time for another definition.  "Message digest" or "hash" are synonymous phrases for what you get when you put one or more numbers into some mathematical function.  We've already met a hash, even though I've not called it that.  Recall that our challenge algorithm takes two numbers, the account password and the nonce, and produces a response.  Stuffing a nonce of 7 and a password with numeric value of 143 into my "hash function" — the fancy name for dividing the password by the nonce and taking the remainder — results in a hash or message digest of 3.  Notice the fact that this isn't trivially reversible; all you know is that the system fed a "7" and an unknown password into some hash function and got back a digest of "3."  It's not immediately obvious how you'd figure out that the number stuffed into the hash function with the "7" was "143."

Now let's return to the question of whether or not to reveal our network's hash function to the world in general.  A hash function could be any mathematical formula that takes one or more numbers as inputs and produces one number as an output.  For example, I could have built a hash function that says, "add the nonce and the password to produce the hash."  That'd be a pretty lousy hash function, however, as anyone who sniffed an authentication would could see first the nonce (7) and then the hash response (150).  Subtracting the nonce from the hash would yield the password of 143 — not good.  Reiterating what I said before about leaving the algorithm design to a publicly-discussed process conducted by mathematical geniuses has worked pretty well and produced a few commonly-used hash functions.  

One such function is called the Data Encryption Standard or DES.  DES encrypts a message, and takes two inputs:  a 64-bit message, and a 56-bit key.  DES then uses the 56 bit key to encrypt the 64-bit message.  (If you want to encrypt a message larger than 64 bits, DES just chops it up into 64-bit pieces and encrypts them one at a time.  Recall that any text message in a computer can be represented as some large number.)   DES has been around for a while, though, and at this point common computers are powerful enough to crack DES-encrypted messages, so there's a variation called triple DES, which as you'd imagine encrypts the message, then encrypts it again, then encrypts once more, making it harder to crack.  DES is designed to be an encryption method that allows decryption; if I knew what key was used to encrypt the message in the first place, then I could use that key to decrypt the message.  According to some sources, NTLM and its 1980's predecessor, LM authentication, convert the user password into a 56-bit key and then use that to encrypt the nonce, and then use the encrypted nonce as the response to the server's challenge.

In contrast, there's a family of hash functions from RSA inc, called MD2, MD4, and MD5.  (MD stands for "message digest" in all three cases.)  These do not encrypt a message so that it can be decrypted, as DES does.  Instead, they take a number of any length — the "message" — and create a 128-bit hash from it.  (See http://www.rsasecurity.com/rsalabs/faq/3-6-6.html for more information on the MDs.)  There isn't any straightforward way to take that 128-bit hash and decrypt it into the original message, and so the MDs are known as "one-way hash functions" or OWFs.  Recall that I said that it's hard to get complete, reliable information on NTLM, and here's an example.  Where some sources say that NTLM uses DES to create a response to a server challenge, others say that NTLM combines the password and the nonce and then hashes it with MD4 to create the response.  Apologies, but I don't know which is correct.  Personally I'd guess that MD4 is what Microsoft uses, but inasmuch as Microsoft doesn't let me peek at the source code, I couldn't say for sure.  I'm going to assume that it's MD4 for the rest of this discussion.

Let me finish this algorithmic discussion by showing how our friend Joe Dirtbag might exploit the incredibly terrible design of my hash function to figure out a password.  Let's assume here that Joe knows our hash algorithm — the server hands the client a number, and the client must then view the password in numeric form and divide the password by the number, returning the remainder to the server.  So he knows that the client took the password — which hasn't been transmitted on the network, recall — and divided it by 7, and came up with a remainder of 3.  Does Joe D have the client's password yet?

No, clearly not.  All the sneak knows is that it's a number equal to some exact multiple of 7, plus 3.  That could be any number of values:  3, 10, 17, 24, 31...; actually, an infinite number of values.  But he's got a clue.  And he keeps listening.  So perhaps later in the day, the client and a server talk like this:

Client: Let me in.

Server: 11

Client: 0

Server: Okay, you're in.

Notice that the server used a different nonce in this second authentication — 11, not 7.

Can Joe Dirtbag guess the password now?  Well, he knows that it's a number that's exactly divisible by 11, and is also equal to a multiple of 7, plus 3.  The password must be a whole number, not a fraction, so he couldn't just set up a couple of equations and solve for them.  But he's got a lot more clues, and bit of figuring shows that the first number that meets that criterion is 66, the next 143, and so on; just keep adding 77 to the previous number and you get the next candidate.  There are only 13 candidates in the first 1000 numbers.  Time to start getting nervous...

You can see where this is going.  Suppose Joe sniffs another authentication challenge — the server says "23," and the client responds, "5."  So the password is a number that's a multiple of 7, plus 3, it's evenly divisible by 11, and is a multiple of 23, plus 5.  A few lines of VBScript let me quickly check the first 1000 numbers to find that the number of possible candidates is ... oops ... one.  143 is the only possibility in the first 1000 numbers.  Just a few sniffs, then, provided enough clues to nail down a password.

But how likely is it that someone sniffing your network would get to see a lot of authentication challenges?  Plenty likely.  You don't just log on once in the morning; your workstation handles dozens of logons for you every day.  And if you run an XP desktop, then you may be surprised to hear that XP does a whole BUNCH of authentications.  Apparently XP periodically looks around for nearby systems and pre-negotiates connections to their shared devices, when there are any.   Those systems ask XP to prove that they should answer XP's question, asking XP to log on.  XP uses your user name and password to log on, and gets the information.  Thus, an XP system spends a lot of time responding to challenges with your account, tossing around responses that implicitly contain your password.  What's that you say, you find that a bit troubling?  Then go to any folder and choose Tools/Folder Options/View and clear the check box "Automatically search for network folders and printers."  (Or look in Knowledge Base article 320138.)

In any case, the moral of the story is that lousy hash algorithms are really bad.  Worse news is that DES is crackable at this point (it takes days to do it, but it's still possible) and MD4 is apparently broken as well, at least according to RSA.  Another reason to move up to more complex authentication algorithms, as you'll see.

Strengthening Challenge-Based Authentication with Big Numbers

But can you see how we'd strengthen even this mega-lame system of mine?  I said that a check of the first 1000 numbers turned up one candidate.  But who'd use a system where you could have one out of only one thousand possible passwords?  What if instead there were trillions or octillions of possible passwords and nonces?  Well, in that case, my VBScript program, which can test 1000 possibilities in a few seconds, would be stymied.  Recall that Microsoft nonces are either 64 or 128 bits, depending upon whom you ask.  Either way, that's a lot of bits, and a lot of possibilities.

But once you get a good algorithm, and nonces that are large, random, and never-repeating, then do you have a truly secure authentication system?  No.  There's one more piece, recall — the password.  No matter how good your security system is, if I can easily guess your password, then you don't have security.

NTLM's Twisted Forebear: LM

So we've seen that Microsoft's NTLM authentication system uses challenges so that it needn't send the password over the wire, employs long challenge tokens so it'd take a long time to try every possible challenge token, one at a time, and creates responses with a hard-to-reverse hash algorithm, MD4.  You'd guess that as long as you pick good passwords, then Microsoft security must be pretty hard to crack, eh?  Well, yes, except for one thing.  Even if you do choose a good password, then it might well be crackable due to a bit of vestigial software —  LM.

LM is the old LAN Manager version of NTLM.  It basically works the same way NTLM works, with a few exceptions.  The big ones are this.  First, it stores passwords in two separate seven-byte chunks rather than a single 14-byte chunk.  (It does that because seven bytes is 56 bits, and they use DES which, you may recall, uses 56-bit keys.)  Why's that important?  Several reasons.  First, if you have an account password that's longer than 14 characters, then LM doesn't care, it just truncates it.  So much for those long, hard-to-guess passwords.  Second, LM stores all passwords in uppercase.  That makes guessing passwords far easier.  For example, if you were running an old LAN Manager system and you chose a password of "cAt," the system would just convert it to "CAT" and store it that way.  No matter whether you tried to log in with a password of cAT, CAT, cat, CAt, or whatever, the system would just convert it to CAT and try to match it to the stored LM version of the password, which is always stored in all uppercase.  Third, cracking seven bytes of a 14-byte password is trillions of times easier than cracking a 14-byte password.  Sort of like the difference between a thousand square miles... and a thousand miles square.

Getting back to this LM issue, though, you may be thinking no problem, as you're not running LAN Manager, right?  Wrong.  Yes, NTLM is smarter than LM and stores your passwords in whatever case you type them in without truncation —  create a password "00cAtINtheeeHat-*%" and that's what you'll need to type to authenticate; NTLM-based systems — that is, every OS in the NT family — do indeed store passwords of up to 128 characters.  (NT 3.1 to NT 4.0's User Manager only had space for a 14-character password, but third party tools could allow you to use longer passwords.  Windows 2000 and later leave space for longer passwords.)

But way back when NT 3.1 first appeared, Microsoft figured that there would people who were using LAN Manager (you remember, all five of those folks) and who would like to migrate to NT 3.1, but to do it gradually, by adding a few NT boxes to the LAN Man network.  Now, the NT boxes used NTLM and the LAN Manager boxes used LM, and that could have meant that they didn't understand how to authenticate with each other.  So Microsoft decided to make NT bilingual, and so taught NT how to do LM authentications.  In order to make that easier, NT 3.1 stored two copies of your password —  the NTLM version, with uppercase and lowercase, and the LM version, all shifted to uppercase.  You didn't need to flip a "make me LAN Man compatible" switch, nope, nor did your NT 3.1 system detect LAN Man systems automagically in some way and only then create the lamer LM passwords.  No, it just created and stored the NTLM and LM versions of your passwords locally in the SAM file.  And if some system tried to log onto your NT 3.1 server to use a shared file or printer, then the NT 3.1 server would really prefer to do an NTLM-style authentication.  But just on the off-chance that one of those LAN Manager boxes wanted to authenticate, then NT 3.1 systems stood ready to do the simpler LM authentications.  As a matter of fact, part of any authentication is the negotiation phase, where the client could say, "I'd really like to log onto you, but I only know how to do LM authentications," and by default the NT 3.1 systems would say "sure, no problem."

Thus, a dirtbag trying to impersonate you could write a program that tried to log onto your NT 3.1 network by trying a bunch of passwords, and he could make life a lot easier on himself by asking the NT 3.1 system to do LM authentications.  As all LM passwords are uppercase, he wouldn't have to try anything uppercase, staggeringly reducing the number of possible passwords to try.  Also, you'd do LM authentications now and then just because if a server is slow then it might not respond to your NTLM authentication request, and so your workstation would automatically drop back to LM.  Anyone listening would see an LM challenge and response which, again, wouldn't be the password — but it'd be a whole lot easier to use that challenge and response to either narrow down password possibilities or perhaps crack the password altogether.  Or the dirtbag could create a member server that collected logons from your workstation to pass along to a domain controller, but that member server would always insist on LM authentications.  All perfectly valid ways to gather more easily-cracked LM authentication information.

Man, those NT 3.1 systems were pretty insecure, weren't they?  Good thing we run state-of-the-art networks with NT 4, Windows 2000, XP and 2003, eh?

Well, actually, there's some bad news there.  You know that LM backward compatible thing?  It's, um, still there.  Even if you've got an Active Directory in native mode, then your DCs and other servers are ready, willing and able to not only do standard Kerberos-style logons, but also to do NTLMv2, NTLM, or LM authentications.  And yes, your Active Directories and SAMs (on your NT 4 DCs or NT, 2000, XP or 2003 member systems) do still include the LM versions of your passwords.  Yikes.

De-Clawing LM

And let's talk about those stored passwords for a moment.  Your account's password is actually never ever stored on a hard disk anywhere.  Instead, when you type a password into any of the security dialog boxes then the operating system runs it through a hash function and stores the hash value, rather than the password.  These passwords are sometimes called the "OWF passwords" or simply "the OWFs" because they're hashed with MD4, a one-way function hash.  That hash is then encrypted, and that's what's stored on the hard disk in the SAM file.  So your password is twice protected — first hashed, then encrypted with DES.  Sounds good, but I did mention that DES can be cracked nowadays, didn't I?  Someone getting ahold of the SAM on your workstation or servers, then, could instantly get the MD4 hash of your password ("the OWF," recall) using a tool called pwdump or its follow-ons, pwdump2 and pwdump3.  Once they've got your OWF, then they can run password cracker programs like L0phtcrack and potentially learn your password.  So make sure those domain controllers are locked up!  You can also use a tool called Syskey to replace the DES encryption with another encryption method called RC4, making SAMs a bit less crackable.  RC4 is used in SSL for secure Web transactions; read more about it at http://www.rsasecurity.com/rsalabs/faq/3-6-3.html .)  I warn you, though, Syskey is a pain in the neck in that it password-protects the SAM and requires you to type the password in every time you reboot the system.

But to return to LM, it presents two potential security holes.  First, any time your system tries to authenticate to another system using LM rather than NTLM, then you expose the LM OWFs (also called the "LM hashes") of your password, where they can be snatched and decrypted offline.  Second, if you let someone get physical access to your computer or if they get administrator-level access over the network, then that person can steal your entire SAM (and, I'm told, your Active Directory, although I've not seen that done) and, and again, decrypt your account passwords offline.  Those things are theoretically possible for NTLM hashes as well, but NTLM hashes are 14 or more bytes long, and so that's a lot more computationally difficult, at least at the moment.  (When the one terahertz chips come out, that might change.)

Removing the LM security hole, then, involves two goals.  First, you want to make your servers — and recall that basically all systems are servers in a Microsoft network, even the workstations — refuse to serve LM authentications.  Second, you want those old easily-cracked LM passwords out of your SAMs and Active Directories.

Could this cause a problem of backward compatibility?  As far as I can see, stomping LM doesn't cause problems for anyone but dirtbags.  I suppose there might be the odd network appliance that's so old or so badly designed that it uses LM authentications, but in my experience I have never missed LM, once it was gone.  Let me stress, though, that I've been running my network like this for a few months, but the network only includes Windows 9x, NT 4, Windows 2000, XP and 2003 systems.  I don't have any Samba set up, any NAS devices, any OS/2 systems, or the like.

In fact, I'll recommend that you take things a step further, and shut off not only LM but NTLM, and only use NTLMv2 (and of course Kerberos if you're running Active Directory).  NTLMv2 improves logons in several ways.  First, as you've already read, NTLMv2 never sends the same nonce twice, making replay attacks impossible.  Second, it hashes passwords with MD5, a more secure one-way function than MD4.  Third, NTLMv2 supposedly makes snatching password information from authentications altogether impossible, given the current level of computer hardware.  Fourth, it time-stamps information, further thwarting replay attacks.  That, by the way, makes NTLMv2 very time-sensitive.  Make sure that your systems are all synchronized to within 30 minutes of each other.  (Time zones count, so don't worry — a system in Chicago that thinks it's 10 AM is perfectly synchronized with a New York system that thinks it's 11 AM.)

Moving to NTLMv2 requires very little work and if you decide that you don't like your network in NTLMv2-only mode, then changing back is very easy.  But in any case, DO get rid of LM.

You can decontaminate your network of LM and NTLM, leaving only NTLMv2 and Kerberos, in four steps:

  • Install NTLMv2-aware software on your clients.  Windows 2000 and later don't need any changes.  NT 4.0 needs a Registry hack.  Windows 9x and ME need you to install the "DS client" and to change the Registry.  You must install the DS client even if you only want to get rid of LM.
  • Tell your servers to reject LM or NTLM logon attempts.  You'll do that with Registry hacks on NT and a local or domain policy on 2000 and later.
  • Tell your servers to stop storing the old LM version of the password.  Not possible on NT 4, but possible with 2000 via a Registry hack, or a policy on XP and 2003.
  • Finally, you've got to have your users change their passwords to flush out the old LM passwords.

Getting Windows 9x/ME Ready

First, get the clients ready and let's start with the old folks... the Wintendo crowd, Windows 95, 98, 98SE, and ME.  They're completely clueless about NTLM, doing their logons with LM.  You can fix that with the DS Client for Windows, a free program that comes with Windows 2000 Server and that enables Windows 9x and ME systems to do NTLMv2 authentications.  The DS Client also allows your Wintendo boxes to get site-awareness (helpful with logons and fault-tolerant Dfs), gives them the ability to search the Active Directory, and access and modify Windows Address Book data.

First, make sure that the Wintendo box knows about the domain.  Right-click Network Neighborhood and choose Properties.  Then right-click "Client for Microsoft Networks" and again choose Properties.  Check the box labeled "Log on to Windows NT domain."  In the field labeled "Windows NT domain:," fill in the NetBIOS name of your domain, not the DNS name.  For example, you'd fill in "ACME" rather than "acme.com."  Click OK twice and you'll get the "rebuilding driver information database" dialog box.

Now you're ready to install the DS Client.  It's on the Windows 2000 Server CD at \Clients\Win9x\dsclient.exe.  Start it up and it'll install the client.  Reboot and start up Regedit.

Go to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control and create a whole new key — a folder — named LSA.  In that key, create a new REG_DWORD entry named LMCompatibility and set it to 3.  Close Regedit and reboot.

You can now log onto the Active Directory from a Windows 9x system using NTLMv2.  Respond to the logon dialog box with separate name, password, and domain; jane@bigfirm.biz would log on as name=jane, password=whatever, and domain=bigfirm, rather than name=jane@bigfirm.biz and password=whatever.

Getting NT 4 Systems NTLMv2-Ready

Next, teach your NT 4 boxes to eschew LM and NTLM in favor of NTLMv2.  Your NT 4 box must be running SP4 or later to be able to do this.  Open Regedit and go to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\LSA and add a new REG_DWORD entry, LMCompatibilityLevel — notice it's LMCompatibilityLevel, not LMCompatibility as in the Wintendo case — and set it either to 4 (reject LM, accept NTLM or NTLMv2) or 5 (reject LM and NTLM, only accept NTLMv2).  

Next, go to HKEY_LOCAL_MACHINE\System\CurrentControlSet\control\LSA\MSV1_0 and create two new value entries, NtlmMinClientSec and NtlmMinServerSec.  These control the minimum acceptable level of security that the client and server pieces of your NT system will accept.  Look at Knowledge Base article 147706 for all of the details, but if all you want to do is to require NTLMv2, then enter the hexadecimal value 0x00080000 for both.  Then reboot the system.  (If all you're concerned about is ensuring that your NT 4 clients use NTLMv2 in talking to Windows 2000/2003 domain controllers, then the LMCompatibilityLevel setting is all that you need, in my experience.)

Let me stress again that I've not had any trouble with NT 4 systems using NTLMv2, but there is a KB article 236414 that says that some people have had trouble, so take a look there if things don't work.

Telling the 2000, XP and 2003 Systems To Only Use NTLMv2

Granted, having to install software and hack the Registry on the Wintendo boxes wasn't fun, nor was hacking the Registry on the NT 4 systems, although you could have created a custom system policy to accomplish that as well.  For the 2000 and later systems, it's much easier; use a policy.

In the Group Policy editor, look in Computer Configuration/Windows Settings/Local Policies/Security Options, then "Network Security: LAN Man Authentication Level" on XP or 2003, or simply "LAN Man Authentication Level" on 2000.  Choose either "Send NTLMv2 response only / refuse LM" or "Send NTLMv2 response only / refuse LM & NTLM," if you can.  If you've got an Active Directory, then make it a domain-wide policy.  If not, then you'll have to modify local policies on your systems singly.  No reboots necessary, just force a policy refresh either with 

secedit /refreshpolicy machine_policy

for Windows 2000 systems, or 

gpupdate /force

for XP and 2003 computers.

Getting Rid Of the LAN Man Hashes

By now, your network is running merrily along with NTLMv2.  But those darn easily-cracked LM hashes are still sitting in your Active Directory and local SAMs.  Time to eliminate them.  You've got to do that in two steps.  First, you tell the NT, 2000, XP or 2003 system to stop creating the LM hashes in the first place.  But that only keeps the system from creating new LM hashes; it doesn't clear the existing ones out of the SAM or Active Directory.  To do that, you've got to change every user's password.

There is not, unfortunately, a way to tell your NT 4 systems not to create LM hashes.  But if you don't have any NT 4 domain controllers, then you're in good shape, as there shouldn't be many local accounts on your member servers and workstations.

On Windows 2000 systems, open Regedit and navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa.  Then create a new key — again, a key, with a folder icon, not an entry — named NoLMHash.  Don't put any entries in it, just create the key.  Reboot and it'll never create another LM hash.

On XP or 2003 systems, use a policy.  Look in Computer Configuration/Windows Settings/Local Policies/Security Options/Do not store LAN Manager hash value on next password change and enable the setting.  Or, if you like you can alternatively create a Registry entry; open Regedit and go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa.  Now, with 2000, you create a key named NoLMHash, but in the case of XP or 2003, you create a new value entry inside the Lsa folder named NoLMHash.  It's a REG_DWORD, set it equal to 1.

Is It Worth It?

This has been a long article and thanks for staying with me, I hope it's been useful.  I wanted to do two things: first, to blow away some of the smoke about how NT authenticates, and, second, to highlight a weakness and a solution for that weakness.

The fact is that LM hashes represent a real bonanza for any internal attacker.  I think you don't hear people talk about them much because it can be uncomfortable to discuss security changes that are really only necessary because we don't trust our co-workers, but unfortunately we face more security challenges from insiders than outsiders.  That's not to say that everyone in our organizations are dishonest, far from it; rather, it's that when we do get a rogue insider, then he or she can do a lot more damage than an outsider.

I personally think that the LM "hole" is one that Microsoft should have plugged a long time ago through their defaults, but they haven't, probably because so many clients use Wintendo boxes.  With hope we'll see LM just a bad memory soon, though.  I urge you to seriously consider rolling out this change and let me close this by offering an performance incentive to go "all NTLMv2:"  logons are faster.  If you've ever read my pieces on how much faster NET USE commands become when you shut off NetBIOS, then you probably wondered why they got so much faster.  I never knew either, but since shutting off NTLM and LM, I've noticed much, much snappier response from my NET USE commands.  I still don't know why, but now I've got a guess:  getting rid of NTLM and LM just plain simplified the logon process.  As the clients and servers have fewer options, things just happen more quickly.  I guess that's why shutting down NetBIOS made things faster, as eliminating NetBIOS kills LM, NTLM, and NTLMv2.

Linux Book Errata

The second edition of my Linux for Windows Administrators has been selling well and has gotten a lot of very nice comments.  (Many, many thanks to those of you who've taken a moment to post a few words on Amazon.  It means a lot.)  But nothing's perfect, so five of you have been kind enough to collect your lists of errata and send them to me.  Admitting mistakes is (wince) always hard, but here goes...

Reader Kevin Rice offers this:

pg. 437 -- Configuring an SMB Server
The text, "Next, restart Samba: /etc/rc.d/init.d smb restart" is missing a forward slash; it should be: "/etc/rc.d/init.d/smb restart". No wonder it didn't work!

pg. 77 -- Getting rid of Linux -- mbrwipe
You wrote, "...type mbrwipe. In an instant, you'll get your C:> prompt back...". I believe that should be an A:> prompt

MBRWIPE:  Kevin wanted to add warning statements to my disk wipe-clean program: (1) Open Notepad and paste in the text below to create a text file called mbrwipe.txt (do not include the lines). (2) Open a console window in the same directory as mbrwipe.txt and type the command:
debug mbrwipe.com <mbrwipe.txt
========== [mbrwipe.txt] ==========
f 200 l 200 0
a 0100
mov ah,9 ; DOS display string function
mov dx,0140 ; point to warning message
int 21 ; call DOS
mov ah,8 ; DOS read character function
int 21 ; call DOS
cmp al,21 ; compare to '!'
je 0111 ; jump if '!' pressed
int 20 ; exit if not '!'
mov bx,0200 ; point to block of zeroes
mov cx,0001 ; cylinder 00, sector 01
mov dx,0080 ; drive 80
mov ax,0301 ; BIOS write disk fn, 1 sector
int 13 ; call BIOS
mov ah,9 ; DOS display string function
mov dx,1c0 ; point to confirmation message
int 21 ; display message
int 20 ; exit

a 0140
db "MBRWIPE - DELETES ALL PARTITIONS ON DISK 80 (C:)?"
db 0d,0a
db "Press '!' to DELETE, any other key to exit. $"

a 01c0
db 0d,0a
db "MBR DELETED.$"

r cx
300
w
q
; Include this line to make sure you get the CR/LF in!
========== [end mbrwipe.txt] ==========

Reader John C. Crichton IV offers these suggestions:

I did notice a few things that I wanted to point out so that you could change them before the next edition.  I don't know that the page numbers will be all that helpful, so I will try to be specific about where they occur.

On page 212, in Chapter 6, when you are discussing chmod, you have a command "chmod o=r speech.txt"  In the paragraph that follows, the fourth sentence reads "The plus sign..."  I believe that this sentence should begin "The equal sign..."   

On page 336, in Chapter 8, when you are discussing Apache, at the top of the page there is a section header titled "Starting/Stopping the Apache Server".  In the first sentence of the paragraph that follows, you use the term DNS server rather than Apache (or HTTP) server.  Looks like a cut and paste error.   

On page 363, in Chapter 8, when you are discussing Sendmail, in the Cookbook Summary, there is a line that reads "Generate a new /etc/sendmail.cf:"  The command that follows is incorrect, it should read m4 /etc/mail/sendmail.mc > /etc/sendmail.cf (instead of mt ...)   

On page 437, in Chapter 9, when you are discussing Samba, near the bottom of the page is a line that reads "Next, restart Samba:".  The command is missing a "/" between the init.d and the smb, it should read "/etc/rc.d/init.d/smb restart."


Reader Ray Lewis offers this suggestion:

The problem (and benefit) of Linux is that it changes so fast. I couldn't wait to dive into compiling my new kernel, linux-2.4.17. I am currently running RH Linux 7.2 with the linux-2.4.7-10 kernel. Every step in the book worked perfectly until I got to the very last one -- Type "lilo" to write the changes to the lilo.conf file. 

Lilo errored out with the following message:             

Fatal: geo_comp_addr: cylinder number is too big (1074 > 1023) 

I finally figured out the various switches to use, -C for lilo.conf.anaconda and such, but then realized that I am using Grub. I include all of that just to let you know that you may want to post an addendum.


Reader Dave Page made this suggestion:

Make sure (if you haven’t found it already, and are willing to accept my humble advice) that you include this link:  www.rdesktop.org.  Very cool stuff, and works like a charm even connecting to an XP Pro box. [Dave is sharing info about a Linux client for Windows Terminal Services and XP's Remote Desktop.]


Finally, Michael Horowitz COMPLETELY humbles me with these comments:

Page 43:

--In reading about the fips utility, I was wondering what file systems it supports for resizing partitions

--You say you can't go wrong with Partition Magic. In general this is true, but www.computergripes.com/PartitionMagic.html (my web site) shows a range of problems with it. There is also a very important FYI for resizing partitions: run scandisk/checkdisk first, then defrag the partition, then run the Partition Magic error checking (Operations -> Check for errors in v6 under Windows) before resizing a partition. The idea being make sure Windows is happy with the file system, then make sure PM is happy with the file system. The last thing you want is for Partition Magic to find some error while it is resizing a partition. You might consider advising readers of this.

Page 52:

Running Red Hat Setup's Automatic Partition Option topic

--It should be pointed out that unallocated space (your term) is the same thing as free space (Red Hat term used in third option).

--I think there is an error in the last paragraph on the page. If you already have an OS on the computer that you want to keep and there is sufficient unallocated space, then the option you want to chose is "Keep all partitions and use existing free space", not "Remove all Linux partitions on this system"

Page 53:

--I think the option to place LILO in the MBR (discussed in the first paragraph on the page) is dangerous and you should note the danger. If there is a pre-existing OS on the machine the possibility exists that this can wipe it out (it happened to a friend of mine). Also the MBR is a single point of failure (I know of no utility to back it up), and as such, all changes to it should be minimized.

--Also in the top paragraph you refer to the "/boot" partition. I suspect there is a space missing, as previously the boot partition is referred to as the "/" partition.

Network and Firewall Configuration topic

--The second paragraph of this topic discusses configuration options for TCP/IP and NIC cards. There is nothing about netbui. As a reader, I was wondering if Red Hat 7.3 supports netbui over Ethernet.

Page 85

Tip at the top of the page.

--I think there is a typo in an error message: "Can't open /dev/fdo0: Device or resource busy" It looks like the letter "o" was typed first and then a zero that was meant to replace the letter.

Mounting a CD-ROM topic, first paragraph

--There is only one type of data CD, but there are also audio CDs. I suspect this paragraph was only written with data CDs in mind. As a reader I was wondering what file system is used on an audio CD and whether Linux can read audio CDs (everyone knows Windows can).

Page 153

--The second paragraph refers to the "-l" options of the "ssh" command. It is not clear if this is the letter "L" or the number one.

--You note that the ssh command assumes you want to log in to the remote machine with the same username as you are using on the current system. You don't discuss the password used when logging on to the remote system. Where does it come from?

Page 235:

--At the bottom of the page it might be helpful to point out that you must type an upper case "X", not a lower case "x". Readers accustomed to Windows will struggle with case sensitivity on Linux for quite a while.

Page 236:

Basic Useful X Introducing xinit and xterm topic

--I think the "xinit" command has to be entered when X is NOT running. As the text reads it appears that the user should enter it when X is running.

--Figure 7.1 looks like a black box, it printed too dark.

Page 244:

--There seems to be a typo in the middle of the page. It says "so to start xterm, I'd add the following line" and the following line starts xcalc, not xterm.

--I think the last example of invoking xcalc also has a typo. This is the example near the bottom of the page with the -geometry option. It does not have the "&" symbol to run xcalc in the background which the text just said to always use.

Page 248

--In the discussion in the middle of the page about running xterm with a blue background and yellow foreground, the text does not match the example. The example has blue as the foreground and yellow as the background.

Page 250

--I think there is a typo in the example in the middle of the page of running xedit with the adobe times font. It reads xedit -fn - adobe-times-medium.... My guess is that the dash between "fn" and "adobe" should be flush against the word "adobe" with no space separating them. On page 251, there is an example of the xfd command that also has the font (-fn) option. It shows the word "adobe" preceded immediately by a dash.

Page 257

--There is an obvious word missing in this sentence from the first paragraph: "...early '80s-it wasn't expensive, but it hardly high-quality"


My sincere thanks to Kevin, Michael, John, Ray and Dave.  I really appreciate it, guys!

Conferences

I hope you'll join me for a seminar but if you can't attend a class then please consider attending another show:

NetIQ Active Directory Webcast May 7

Visit http://www.netiq.com/events/webcasts/activedirectory/default.asp to sign up for my free webcast on running AD with nary a hitch. 

FREE — Windows Decisions Chicago May 14-16

Once again TechTarget delivers a Windows 2000/XP/2003 conference with excellent content... free.  Last year's show featured a whole bunch of great speakers on a wide variety of topics and, of course, the price is right, if you qualify.  Visit http://windowsdecisions.techtarget.com/html/registration.htm?Offer=wdmkmn1 to apply and we'll see you in Chicago!

Windows Magazine Live! May 18-21 in Phoenix

The magazine that I write for, Windows and .NET Magazine, holds its next Windows Magazine Live! conference in Phoenix this May.  It's a jam-packed set of great talks by some great speakers including of the industry's foremost megacephaloids like Mark Russinovich, Intel's Sean Deuby, IIS Answer Man Brett Hill, Uberscripter Bob Wells and more — great speakers all and really smart guys.  I'm also doing three talks, including two new ones:  "How To Troubleshoot Any Network Problem" and "The 2003 Report Card," as well as my "Tuning XP, 2000 and 2003" talk.  Watch www.winconnections.com for more info.  The Phoenix site is always great, don't miss it.

Bring Mark to your site to teach

I'm keeping busy doing Windows Server 2003/2000 and XP seminars and writing, but I've still got time to visit your firm.  In just two days, I'll make your current NT techies into 2000 and/or 2003 techies.  To join the large educational, pharmaceutical, agricultural, aerospace, utility, banking, government, transportation, and other organizations that I've assisted, either take a peek at the course outline at www.minasi.com/2003outln.htm, mail our assistant Jean Snead at Assistant@Minasi.com, or call her at (757) 426-1431 (only between 9-5 Eastern time, weekdays, please).

Until Next Month...

Have a quiet and safe month.     

Please share this newsletter; I'd like very much to expand this periodical into a useful source of NT/2000/.2003/XP information.  Please forward it to any associates who might find it helpful, and accept my thanks.  We are now at over 25,000 subscribers and I hope to use this to get information to every single Mastering XP, NT and 2000 Server reader. Thanks for letting me visit with you, and take care.  Many, many thanks to the readers who have mailed me to offer suggestions, errata, and those kind reviews.  As always, I'm at http://www.minasi.com/gethelp and please join us at the Forum with technical questions at www.minasi.com/forum

To subscribe, visit http://www.minasi.com/nwsreg.htm. To change e-mail, format, etc., link to http://www.minasi.com/edit-newsletter-record.htm.  To unsubscribe, link to http://www.minasi.com/unsubs.htm. Visit the Archives at http://www.minasi.com/archive.htm. Please do NOT reply to this mail; for comments, please link to http://www.minasi.com/gethelp.

All contents copyright 2003 Mark Minasi. You are encouraged to quote this material, SO LONG as you include this entire document; thanks.