Cisco PIX/ASA Causes SMTP Banner Corruption

Traffic inspection rules on a Cisco PIX or ASA firewall will sometimes cause the SMTP banner to appear corrupted.

When testing access to your mail server from outside, you may notice that the SMTP banner looks like this:

This is just a symptom of the problem, which is that the SMTP traffic inspection rule is interfering with the SMTP data stream.  Another symptom would be to see email messages destined for this server seemingly stuck in the SMTP queue on a server outside the network.  This can ultimately cause delayed and undeliverable mail, especially for larger messages, such as those with attachments.

The resolution for this problem is to disable the traffic inspection rule for SMTP/ESMTP on the Cisco PIX or ASA firewall.

On a PIX, this can be done from the command-line using the “no fixup protocol SMTP 25” command.  It can also be disabled from the PIX Device Manager (PDM).

On an ASA, it’s a little different.  From the command line (assuming your policy map is named “global_policy” and your class is named “inspection_default”):

CiscoASA(config)#policy-map global_policy
CiscoASA(config-pmap)#class inspection_default
CiscoASA(config-pmap-c)#no inspect esmtp 

From the Adaptive Security Device Manager (ASDM):

1.       Go to Security Policy –> Open the inspection rule:

2.       Go to the Rule Actions tab and uncheck the box next to ‘ESMTP’

3.       Test from outside the PIX/ASA again by telnetting to port 25; your SMTP banner should now look like this (I have masked the name of the server for privacy).

That’s it.  I have made it standard practice to just disable this inspection rule on all Cisco ASA firewalls that I deploy to avoid problems.

Posted via email from Aaron Johnstone

Supporting Exchange 2007 on Windows Server 2008 R2

While it was previously announced that Exchange 2007 would not be supported on Windows Server 2008 R2, that decision has been reversed and support for this combination will be forthcoming.

According to this post, at the Microsoft Exchange Team Blog:

We always talk about listening to customers and sometimes this is written off by many as ‘marketing speak’.  In fact, we do take feedback seriously and no input is more important to our engineering processes than your voice.

Earlier this year we made a decision in one direction, and due to the feedback we have received on this blog and elsewhere, we have reconsidered.  In the coming calendar year we will issue an update for Exchange 2007 enabling full support of Windows Server 2008 R2.  We heard from many customers that this was important for streamlining their operations and reducing administrative challenges, so we have changed course and will add R2 support.  We are still working through the specifics and will let you know once we have more to share on the timing of this update.

So, keep the feedback coming.  We are listening.

Kevin Allison
GM Exchange Customer Experience

Posted via email from Aaron Johnstone

Windows Server 2008 R2 is 64-bit ONLY

Just in case you are wondering.

Here is a technet blog post about it:

…and a little more on the subject at Brian Madden’s blog:

Posted via email from Aaron Johnstone

How to Safely Transfer DNS Records

Occasionally, you will find yourself in a situation where you have to transfer DNS records from one set of name servers to another.  It is not a technically difficult process, but one that must be handled with the greatest care so that you don’t leave your company (or your client) without email or access to their website for hours or days.

What does ‘transfer DNS records’ mean?  All of the DNS records for a given domain (i.e. are managed on the authoritative name servers for that domain.  That means all of the ‘A’ records, MX, TXT, SPF and every other kind of record (except for your reverse PTR!) are stored primarily on a certain group of servers that are authoritative for your domain.  You can find out the name servers for your domain a couple of different ways:

  • Log in to the registrar account where you have registered your domain name (i.e. GoDaddy,  In the control panel or settings for the domain, you can see the name servers.  Unless you changed them, they are likely set to the defaults for your registrar.  For Godaddy, they will be something like and
  • Use an online tool like MXToolbox or Hexillion, which will show you your domain’s name servers and will probably be quicker than having to log in to your registrar account.

There are several reasons why you might need to change those name servers to something else:

  • You need to consolidate a domain name obtained through a company merger into another registrar account that you normally use.  For instance, your company acquires another company who has their domain name registered with GoDaddy, and all of your existing domains are at Network Solutions.  You would actually transfer the entire name, from GoDaddy to your Network Solutions account.  The name server change will actually happen as part of the domain transfer from one registrar to another.
  • You have a hosting company for your website and they “require” that you set the name servers for your domain to point to their name servers.  This is not really true; they don’t HAVE to have control of your DNS records.  All they really need is for your to point the ‘A’ record for and to their web server (depending on what your domain name is, of course).  But that is a little bit of a tangent.  For the purpose of this post, let’s just assume that you are changing your name servers to point to your web host.
  • You need to have a reverse lookup zone and the only way that your ISP will create one for you is if they host your DNS for you, a la AT&T.  This is also not something that you HAVE to do.  Again, kind of a tangent.  So, let’s assume that you are changing your DNS servers to point to your ISP so they can create a reverse zone for you.

If you are in one of these situations, or are just moving your DNS servers around for the fun of it, here is what you should do:

  1. Find all of the DNS records that exist for your domain on the current name servers. If you have access to some kind of ‘DNS Management Console’ in your registrar account, like at GoDaddy or Network Solutions, this is where you need to go.  Record or export a list of all the records that exist and where they are pointed.  The most important records are where the root domain name itself is pointed as well as ‘www’ and of course your MX records.  If your name servers are pointed at something outside the control of the registrar, you won’t have any kind of DNS management console.  In this case, the only way to get a complete list of all the DNS records (aka a full zone listing) will be to contact the entity that manages the name servers that your domain is pointed at currently.  Most likely, this is going to be whomever does your website hosting.  Just contact them and ask for a full DNS zone listing for your domain.
  2. At least 3 days before you change the name servers, change the Time-To-Live (TTL) values of your most critical DNS records to 3600 seconds (1 hr.) or less. The TTLs dictate how long other DNS servers that have looked up your ‘A’ or MX records can cache them.  TTLs are also why you always hear the butt-covering statement or warning that “it can take 48 – 72 hours for DNS changes to propagate”.  A lot of TTL values are set to 86400 seconds (1 full day).  If multiple levels of DNS servers from your name servers have cached these records, it can definitely take 2 – 3 days for any changes to those records to propagate.  So, if you change these to much lower values (3600 seconds), the changes you make will propagate in 2 – 4 hours at most, rather than 2 – 3 days.  Now, if you are just moving the name servers, and are not actually making changes to any of your DNS record values at the same time, then you shouldn’t even have to worry about this.  But better safe than sorry.  If something goes awry, and someone on the other end doesn’t recreate all of your DNS records as you specify, recovery can occur much more quickly if the TTL values are set very low.  I’m just sayin’….
  3. Provide the full listing of DNS records for your domain to the company to which you are transferring the name servers. If you are transferring to another account/registrar that you already control, you can’t really do this (I’ll mention more on that later in this post).  If you ARE transferring the name servers to someone else and out of your control, make sure they get your list and they understand what it is for.  You’d be surprised at how little some people at hosting companies, who deal with DNS records all the time, actually know about DNS.  There are usually some people on the back-end that know what is going on, but sometimes the front line people….don’t.
  4. Verify the DNS records have been created in the new location. If the 3rd party guys have done their job, they will have recreated all the DNS records you gave them and have provided you with at least two server addresses where your DNS records are hosted.  You should use nslookup or dig to verify that each of the DNS records you asked them to create exist and are accurate.
  5. Set Expectations. Whether this is your company or a client, you should set some expectations with regard to what is going on.  This is a very important step.  You don’t necessarily need to explain this whole process and how DNS propagation works, but you should let them know that changes are being made and that the potential exists for disruption of certain things like access to the website and email.  Get contact information for the decision makers and/or your points-of-contact at the company so you can relay information to them in some form besides email in case you run into problems.
  6. Switch the name servers or transfer the domain to new registrar. This is cutover time; start biting your fingernails.  This is where you actually go into your registrar account and switch the name servers from (example) to whatever addresses were given to you by the 3rd party.  Or, you are transferring the domain to another account you control.  You should periodically check to see what name servers are listed for your domain using one of the methods I mentioned at the beginning of this article.  Now, as I said before, if you are just transferring the domain to another registrar and you will have control of the DNS records there, this is the point at which you would go into the DNS management console at the new location and recreate all the DNS records yourself.  If you have any trouble getting into the new DNS management console, call the registrar as soon as possible to resolve any problems.
  7. Test, test, test. Check the name server and do DNS lookups using various methods throughout the evening/weekend after the change until you see that everything has propagated.  Check access to your website.  Send test emails to and from addresses on the domain in question.  If there are any problems, contact the registrar or the 3rd party company hosting the records immediately.  It would be a good idea to have after hours contact info for the people involved so you know you can get a hold of someone in a pinch.

Various ways to reboot a Windows Server remotely

There are many different ways to reboot a Windows Server.  When you spend as much time working on systems remotely as I do, you want to have as many tools/options available as possible when you are working on a server in Virginia (I live and work in Central Texas) and it doesn’t reboot normally.  Instead of panicking, you can try one of these**:

1. Try using remote desktop to access the server.  If you can log in to it, you can reboot it.

2. Connect to the server using computer management from another server/workstation.  From another machine, right click on My Computer and go to ‘Manage’.  When the computer management console comes up, right-click on the top of the hierarchy, which says “Computer Management (Local)”, and choose ‘Connect to another computer…’.  Type in the name of the server you are trying to reboot.  After this, you should be able to right-click on Computer Management and go to properties.  From within this dialog box, you can reboot the server.

3. Use the built-in shutdown.exe program which is included in Windows XP/Vista/7 and Windows Server 2003/2008.  To access it, just run ‘shutdown /I’ from the command line and you will be presented with a window that looks like this:

From here, you can add the server you are trying to reboot, set the options, and then execute it by clicking Ok.

4. Use PsExec; this is a utility created by Sysinternals, an awesome, awesome company that created many very useful tools.  (check them out here).  In this case PsExec is useful because it can be used to execute commands on other systems over the network, such as ‘shutdown’.

5. Use Remote Task Manager.  This has saved my skin on several occasions.  The trial is fully functional, but a single license is only $40, which is not bad.  It copies a file to a remote system, starts a service, and then lets you manage the server, including forcing a shutdown/reboot.

6. Use iisreset.  I was not aware of this until recently,  but yes, it is possible.  I found this one on John Pollard’s blog. (this will only work if the destination server has IIS installed)

iisreset [computerName] /reboot

7. Use a network-enabled power distribution unit (PDU).  These can be very handy devices.  Essentially a power strip with a web interface, these will let you power off and on specific ports on the power strip.  If you have good documentation, you can power a server off and on again.  Not ideal, because this is like using the power button, but better than having to face a potentially long drive to a client office.  They can be had for as little as $300 (maybe less), but I would recommend a slightly higher-end device, especially if you are going to have critical servers plugged into it.

8. Use management software, such as Microsoft SCOM, or other managed services software such as Kaseya.  Tools like these always have some way to initiate remote commands.  In the case of Kaseya, a software agent resides on each server/workstation.  If that agent is still able to check in with the management server, you can use it to run a shutdown/reboot command.

9. Server management cards, such as DRAC (dell) and iLo (HP), will allow you to access the BIOS via web or telnet and reboot the server if the OS is hung.  Of course, you have to have set this up beforehand, or it will do you no good.

10. Network KVMs.  These will allow you to access the server console over then network from a web browser and (usually) a Java applet.

Leave any additional methods you have found in the comments and I will add them to this list.  Thanks!

**keep in mind that some of the more ‘abrupt’ methods listed here can cause data loss if someone has a file open or the server is writing to disk.  These methods should only be used as a last resort or if you KNOW you’ve got a good backup!

Posted via email from Aaron Johnstone

Sizing virtual machine hosts

How do you size a virtual machine host?

There are 4 main criteria which you have to evaluate:

  • CPU
  • Memory
  • Storage
  • Network


Make sure you have CPUresources sufficient to eclipse the most demanding guest VM that you are planning to run on the host.  If you are planning on running a SQL server, get at least a quad-core proc.  Not to belittle this component, but you are unlikely to run into a problem with processor speed, since most processors are severely under-utilized as it is.  The most important thing I can say about processors is to think ahead.  IF, at some point, you think the server you are designing might be part of a VMware cluster, go look at the processor compatibility list for VMware.  Some of the features, such as fault tolerance (FT) require the exact same model processor in all the nodes in cluster.  I’m sure Microsoft Hyper-V has similar restrictions, so if Hyper-V is your preference, check the compatibility guides for Microsoft as well.


The more the better.  Memory is cheap these days.  Get as much as you can afford or as much as the physical server you are installing can support, whichever threshold you reach first.  In addition, keep in mind that with most servers, memory slot population is very important.  Doubly so when you have a dual proc config.  On top of that, VMware ESX 4 is extremely picky when it comes to the memory slot population.  Have you ever heard of a NUMA node?  Well, you don’t want to learn about it when VMware ESX fails to boot because the NUMA node requirements for your server/blade have not been met.  Most likely, if you follow the guidelines for your server on memory slot population, you will also meet the NUMA node requirements for VMware.  At least, that is what  VMware support will tell you when you call them in a panic… 🙂.  Moving along.  If you have a single or dual proc server that has fully populated memory slots, which you should, it isn’t likely to be an issue anyway.


This is the most difficult thing to get right and the hardest thing to fix if you get it wrong.  So…get it right.  For one, you are going to need more storage than you think you will need, unless you are just a gross overestimator.  Here are some of the things you need to plan for:

  • Amount of storage: you are going to need space for your OS, data, snapshots,  pagefiles, and, at least in VMware’s case, .vswp files.  A .vswp file is a file created in a datastore that is equal to the size of the amount of memory in your virtual machine; created when you boot the VM.  Not a big deal when you have a machine with 2 – 4 GB of RAM.  A bigger deal when you have a SQL VM that has 20 GB of RAM in it.  You are  also going to need room to grow.  If you are consolidating a bunch of physical servers, I would recommend adding up the total amount of used space on all of them and multiplying by at least 4.  That way, you have plenty of room to grow, and you have room to keep multiple generations of full VM backups on disk for quick restore if necessary.

  • Type of storage: this depends on what kinds of VMs you are going to be running.  To some degree, you can plan it like you would storage for separate physical servers.  Example:  say you need to put up a SQL VM.  So, that means you are going to need some fast storage, such as that provided by a RAID 10, for the log files.  And you are going to need another volume, on separate disks if possible, for the database files themselves; that can probably reside on a RAID5 or 6.  You will also need a separate volume for the OS, which can probably just be a RAID1.  If you are working on a large virtualization project, you most likely have shared storage, such as a SAN.  So, you might break out two large disks into a RAID 1 config to host the OS disks for several VMs.  Then, you can take 4 disks and carve that out for a RAID10 for high performing storage for your log files.  Then, you can take the remaining disks and partition them into a large RAID5 or 6 for general storage such as  for your database files.   These volumes can all be  shared by many different VMs.  How many will depend on what types of VMs you have and the speed and capacity of your disks.  General guidelines I have come across when learning about shared SAN storage say this:
  1. Shoot for LUN sizes of up to 500 – 600 GB
  2. Use a single LUN to support up to 15 – 20 virtual disks


Make sure you have plenty of network adapters in your physical server.  I would recommend 4 network adapters (ethernet ports) per physical server, and I will use VMware ESX to give an example.  With 4 adapters, you can dedicate one to the service console, which is where all the management traffic is handled.  Another adapter can be dedicated to VMotion.  Finally, you can either set up NIC teaming on the other two adapters in VMware and present them to your VMs as a single NIC, or just pass along two NICs to each VM and have two connections on each server.  I recommend the former, since it avoids you having to set up NIC teaming in your VM for two separate connections.

Also, a couple of websites that I would recommend for virtualization news and information:

TechTarget – SearchServerVirtualization

VMware Support (lots of documentation here, some specific to VMware, some general virtualization design info)

Microsoft Virtualization Calculator – Meeting your licensing requirements in your virtual environment

Thanks for reading!

Posted via email from Aaron Johnstone

NTFS Security Not Transferred When Copying Files

When copying files from one server to another, the NTFS security ACLs on them are not transferred, and the files inherit the permissions of the destination folder.  If the permissions are simple, and set at just one level at the top of the folder hierarchy, it’s not a big deal to just set them again manually.  But if you have multiple folder levels of settings that may or may not be the same, or if you have particularly sensitive data and you want to be sure the security of that data is maintained, here is what you can do.

1.        Transfer the files over using whatever file copy utility you like.  I like RichCopy, which is just a nice, GUI front-end to robocopy.

2.       Even though you have avoided using the command-line for the transfer itself, you are still going to have to use it now to get the file/folder security settings moved over.  For each folder that you transferred using Richcopy, run the following command from the source server:

robocopy "X:\sharedfolder" "\\servername\newshare"  /E /COPYALL /SEC /XC /XN /XO /R:1 /W:0

The "/xc /xn /xo" part of the command excludes files from being copied over again.  The “/E /COPYALL /SEC” switches actually re-sync all the security settings for all the files/folders, so they end up matching the security that is set on the source.

(Robocopy is part of the Server 2003 Resource Kit)

Posted via email from Aaron Johnstone

Set up VPN Client Access on a Cisco ASA

This article walks through how to configure a remote access VPN connection on a Cisco ASA 5500-series firewall.

1.  Log in to the Cisco ASDM
2.  Go to the Wizards menu and run the VPN Wizard.
3.  Choose the ‘Remote Access’ type on the VPN Tunnel Type page:

4.  Choose “Cisco VPN Client” for the VPN client type.
5.  Set your pre-shared key and your Tunnel Group Name
6.  Choose the client authentication method, either using the local user database or an AAA server group
7.  Add a DHCP pool to be used for users connecting using a VPN client; use a different subnet than one already in use on your LAN

8.   Configure the DHCP scope options, such as DNS, WINS, and default domain name
9.   Set the IKE and IPSec policies; I normally use the defaults, which currently are 3DES-SHA
10. Set the Address Translation Exemption and split tunneling options.  Typically, I use the internal network that the VPN-connected client will need access to and enable split tunneling.
11. Click Next, then Finish.
12. Go check the NAT exemption rules.  You should have a rule on the inside interface, exempting any traffic that is going to the VPN subnet (in this case from NAT.  Should look like this:

13. That is it; you are done!  You should be able to set up your Cisco VPN client, connect to the network, and test by pinging one of the servers on the internal LAN subnet.

    Posted via email from Aaron Johnstone

    Configure Cisco ASA remote access VPN to use RADIUS

    This article will help with setting up a Cisco ASA 5500-series firewall to use RADIUS to query a Microsoft Windows Active Directory domain controller to authenticate users who are connecting in using the Cisco VPN client.

    1. Install the Internet Authentication Service (IAS) Windows component

    2. Open the IAS console

    3. Add the Cisco ASA as a RADIUS client

    4. Edit the remote access policy in the IAS console as needed; enable “Unencrypted authentication (PAP, SPAP)” on the Authentication tab of the profile

    5. Connect to your ASA (assuming you are using the ASDM)

    6. Go to the Properties tab, then to AAA Setup à AAA Server Groups

    7. Create new server group

    8. Add a server to the group

    9. Test the authentication

    10. Go into your VPN settings on the ASA (General à Tunnel Group à properties of the remote access VPN)

    11. Go to the General à Authentication tab and change the Authentication Server Group property to the new AAA Server Group that you just created

    12. Check the box to enable LOCAL authentication if the server group fails

    13. Test it with an Active Directory user account from outside using the Cisco VPN client

    How to create a DFS replica in Server 2000

    When you are creating a DFS replica of a shared folder to a new location:

    Create a shared folder on the destination server

    Right-click on the share in the DFS management tool and choose Add Replica…

    Select the shared folder you just created

    Configure the replication to happen automatically.

    After you have created the replica, right click on the new replica and TAKE IT OFFLINE until the replication has completed.  This is the most important step.  If you leave it online, users can and will get directed to the new location, which will appear to be missing files/folders.

    Once you are satisfied that the replication is complete (i.e. the size of the folder on both servers and the number of files and folders match), then you can bring the new replica online.

    Posted via email from Aaron Johnstone