We’ve been waiting on how the contract negotiations with Microsoft will go and then determine through the Microsoft design, planning, and scheduling to see how long it will take.

The problem is that this site relies on a 3rd party host provider. We have no guarantee of backups, no resolve if the company goes bankrupt, no compliancy capturing, and then the added cost of paying for the external service.

Their existing mail setup is Outlook 2003 or 2007 accessing POP3 mailboxes with local .pst files. The meeting was to determine how we should proceed in the next few weeks because the management at that site is now very concerned several months after we had planned on bringing this service in-house that data is not secure and protected.

I explained the issues to Microsoft and we came up with 4 options. We also have these assumptions: no webmail/Internet/BES access necessary, no compliancy capturing necessary, Any VIP users who need webmail/internet/BES can be migrated to a nearby site that uses Domino already.

1. Do nothing now. Wait until they rollout the global Exchange infrastructure and migrate the existing .pst mailboxes to global Exchange environment. Microsoft claims that it will only be approximately, 9 to 10 weeks before they can roll the global Exchange infrastructure out to Site A if they use Site A as one of the first sites. They do not have their schedule put together yet, but the first draft will be ready next week.
User Impact: None now, point Outlook to global Exchange later.
Pros: No user impact now, no labor costs/resources required now
Cons: Continued risk in using the external host provider, no mailbox backups

2. Build a separate AD domain, with temporary exchange environment. We will “cut off” the user’s existing mailboxes (local archive .pst files). We will build 2 virtualized AD domain controllers, 2 exchange 2007 servers, and High availability is provided by CCR (Continuous Cluster Replication). This will require either a 3rd server for the CCR, or a shared storage (SAN).
User Impact: Point Outlook to new exchange server now, point Outlook to global Exchange later.
Pros: Mailbox Backup
Cons: User will have two local archive .pst archive files after migration to global Exchange environment, slow setup, requires either 3rd server or SAN, We must touch desktop now, and later.

3. Build a separate AD domain, with temporary exchange environment. We will “cut off” the user’s existing mailboxes (local archive .pst files). We will build 2 virtualized AD servers, 2 Exchange 2003 stand alone servers, with no high availability (without CCR).
User Impact: Point Outlook to new exchange server now, point Outlook to global Exchange later.
Pros: Mailbox Backup, Quick setup.
Cons: User will have two local archive .pst archive files after migration to global Exchange environment, We must touch desktop now, and later.

4. Turn on the Domino servers we previously built and and turn on the POP3 task.
User Impact: Point Outlook to new POP3 server now, point Outlook to global Exchange later.
Pros: Brings the existing hosted servers in house so we don’t have to rely on hosted provider, clients can be configured to leave mail on server so mail can be backed up, servers are already built so there is a fast turn around, can use the existing .pst file, Mailbox Backup.
Cons: We must touch desktop now, and later.

With options 1 and 4, we can migrate the local .pst file to the exchange mail box in the future (limit 2GB), or it can remain on local drive as an archive.
With options 2 and 3, we must download data manually from interim exchange server to local .pst file (which creates 2 .pst file). We can either leave them local as archives or migrate them to the exchange mail box in the future (limit 2GB).

There was a Microsoft project manager in the room, plus a guy from Microsoft Premier services who was on site to help us with this particular task. Then there was the Microsoft Engineer who is designing the global infrastructure.

The two Microsoft guys (planning engineer and premier) kept going back and forth to determine who was going to have to deal with the issue. Basically, the main point was that it is not simple to migrating from one Exchange environment to another Exchange environment. The planning engineer said that its just as much work to migrate from Exchange to Exchange than it is to migrate from Domino to Exchange. It takes the same amount of planning and design….and thus time. They really did not want us to setup a temporary Exchange environment. They wanted us to do nothing.

In the end, we’ve decided to fire up the Domino server cluster that we built for this site and turn on the POP3 task. The reason we had to fire them up is because we thought Microsoft was going to need them for temporary Exchange servers, so we made Ghost images of them and powered them off to save electricity.

The reason we are having Microsoft plan this is so that when that site actually does get migrated to the global Exchange environment that it will be considered “in scope” and they will handle the migration without claiming that it wasn’t planned for in the scope defining.

P.S. No one here REALLY thinks that they’ll be able to roll anything out in 9 to 10 weeks as they claim.

FOOTNOTES:
Exchange must be 2007 before it can be virtualized. Must meet lots of requirements quote from Microsoft Premier guy, says “it’s not simple.”
Exchange 2007 can only be installed on 64bit compatible architecture.
Exchange 2003 must have a shared storage, otherwise it must be stand alone…you cannot use CCR.
Exchange 2007 requires us to have 3 servers to use CCR. The third servers becomes the transport role. so effectively, you must have 2007 to use CCR, and then you must have 3 servers.
In Exchange 2007 there are 5 roles: Edge, Client Access Server (CAS) or Outlook Web Access (OWA), Unified messaging (UM), transport for SMTP connection, and the 5th role is mailbox

First, let me explain our environment. Most of our company is on Domino and Notes 6.5.1.-6 (mostly Notes 6.5.4).

There is one division that refuses to move to Notes because they love outlook, and their main excuse is that there is the 100 mail rule limitation of Notes. They actually use email as a system like I’ve never seen. They manage a fleet of ships, and each ships sends different kinds of messages, these messages are sent to groups that are nested several times, so many many people receive them. Some people need them for one reason or another. These messages also go into a linux application server and are search able via a web interface.

So basically, they cannot live without having sometimes 200 mail rules. They are pushing electronic mail to the boundaries. I’m sure there is a better way for them to do all of this, but who has the time to pitch a better idea to them, especially when they have no budget.

So, these guys access Domino mail files on 3 servers that we have that serve as both passthru server to everyone in the company, and SMTP/POP3 for this shipping divison. They access the server either with outlook, outlook express, or exchange client via POP3. A few of them are using the DAMO or Outlook connector. It’s a real mish-mash of clients out there. A huge mess and nightmare to support.

I don’t know if you have ever seen POP3 on a Domino server, but I wouldn’t recommend it. The servers are literally so busy, you can hardly issue commands on them sometimes.

We require authentication for SMTP connection so that these guys can send outgoing (from their client) mail.

What happens occassionally is that one or two of their PCs (or more) will get a virus, and start SPAMMING to no end. Since, they are authenticating to these 3 (passthru, SMTP, POP3) Domino servers, they are allowed to route mail through them. The virus, literally serializes the FROM address, so we don’t exactly know which user it is, and have no way of doing anything about it.

The main problem is that it tried to send Internet mail out, and alot of the messages are either blocked, or simply invalid domains. We have an outbound SMTP hub gateway, and initially these shipping users with the viruses were clogging up the outbound SMTP hub gateways, and legitimate Internet mail could not be delivered. So we allowed these 3 servers to deliver mail directly to the Internet, so that only the shipping mail would be affected (we are in a battle to force them to move to Notes client for glaringly obvious reasons).

What started happening in the last couple of days is that there is a particular clever SPAM virus, or malicious person who has control of one of their machines. We receive hundreds of SMTP connections to one of these servers, this in turn takes up so much server resources, that the router task cannot deliver legitimate mail to local Domino mailboxes, or mailboxes on servers sitting in the same Notes Named Network (NNN). Additionally, many of these messages are to invalid Internet domains, so what resources the servers does have left after answering the SMTP connections is to try and connect to invalid or expired Internet domains to deliver these SPAM messages.

We found that there wasn’t much we could do about it. We looked at the from address which looked something like “agds” jgo@ourdomain.com, “bgssd” jgo@ourdomain.com, “zsd;js” jgo@ourdomain.com

This makes it difficult to go through the mail.boxes (we have 4 on each server) and sort by sender and delete all of these rogue messages.

The only thing we can really do is go to the inbound SMTP controls and refuse messages from jgo@ourdomain.com. The issue is that a few hours later, the FROM address will be serialized and it will turn out to be a different FROM address.

Any ideas?

On top of that, our outbound SMTP gateway started coming up with lots of these types of messages:

Router: [0000000C] Skipping message because ReadyToSweep

We had never seen them before, so I looked it up and found this IBM technote:

http://www-1.ibm.com/support/docview.wss?uid=swg21260649

It turns out that our PR/Marketing department had sent out our 2007 annual results. They used a group which contains everyone in the company, and they also have their own personal address book entries with shareholders, business partners, etc. etc.

It turns out that the servers were overloaded trying to send to invalid or expired addresses.

Both of these things were happening at the same time, and I was beginning to wonder if the roof was going to cave-in down us next.