NJ to MD Datacenter Migration

townwebsites

New Member
Any update on the MD migration? Will you be notifying each VPS account when their VPS has been transitioned?

Not particularly happy to tell my clients their New England server is now Mid-Atlantic. Sure hope the hops, ping times etc remain the same or lower.
 

KH-Jonathan

Director of Managed Services
Staff member
Any update on the MD migration? Will you be notifying each VPS account when their VPS has been transitioned?

Not particularly happy to tell my clients their New England server is now Mid-Atlantic. Sure hope the hops, ping times etc remain the same or lower.
The migration is in progress. We do now have accounts running in MD and more accounts are coming online there every day.

Based on our testing, the impact is actually to the positive for most locations and very few will see negative impact from this, and the "negative" impact should only be a few ms in most places.
 

dobee

New Member
I am in southwestern Pennsylvania, closer maybe....maybe the same distance.....
I am not worried, Known Host has it covered. You guys ROCK!
 
Hello good evening.
Come share the information with my friends who use the services of Known Host and also alert the service provider about what is happening to me .
Today is Tuesday and here where I talk ( GMT -3 ) are exact 19:26, my service monitors ( Today I use various services , Pingdom among others ) are sending me alerts regarding the constant interruptions of connection to the server , this for all services , http , smtp , pop among others, when it actually occurs the connection to the server is turned off , given that the server does not respond to ping this period and also numerous customers are entrenando contact me informing about instability of services.
I ran the tracert command at the prompt and I could see that the vps now has more jump " backup01 - nj.privatesystems.net " that before the events really did not .

For all that I have reasons to believe that everything that's happening to me is because of this migration , and unfortunately we were told it would not see hardly human migration , not only humans realized how human these are my customers and are now complaining services and rightly so .

Do not open any call work because I believe this proportion actually instabilities can occur , My post is for aletar colleagues that migration can bring this problem to you too.

Thank you!
 

KH-Jonathan

Director of Managed Services
Staff member
The overall migration is approximately 43% complete. The original post has been updated to reflect this.
 
Last edited:

lyndonr

New Member
I think my VPS is in MD now but what's up with this traceroute?

I went from around 13 hops to 25? Perhaps there some kind of tunneling going on that will eventually be removed.

Thanks in advance!

Screen Shot 2014-03-05 at 2.37.56 PM.png
 

KH-Jonathan

Director of Managed Services
Staff member
@lyndonr,

You're definitely hosted in MD now. The connection is indeed being tunnelled through NJ and routing will change and clean up once things are done.
 

KH-Jonathan

Director of Managed Services
Staff member
Me the same thing is happening.
My monitors are alerting services at any time by informing connection problems.
My uptime is currently 99.876%
We've had a few DDoS attacks which caused some service interruption. This is one of the big benefits of moving to MD is that we'll have much more resiliency to these type of attacks and should significantly increase your uptime :)
 

lyndonr

New Member
Thanks for the quick reply Jonathan. Glad the routing will get sorted out.

All in all I can't say as I saw much more than a few minutes of downtime.. so well done.
 

KH-Jonathan

Director of Managed Services
Staff member
The overall migration is approximately 66% complete. The original post has been updated to reflect this.
 

KH-Jonathan

Director of Managed Services
Staff member
The overall migration is nearing 80% completion. The original post has been updated to reflect this.
 

KH-Jonathan

Director of Managed Services
Staff member
The migration of accounts is completed. We are in the process of removing the network tunnel so traffic will be routed directly to MD. I'll post an update when this is complete.

The original post has been updated to reflect this.
 

KH-Jonathan

Director of Managed Services
Staff member
The tunnel has been removed and traffic is now routing directly to MD. This fully completes the datacenter migration.
 

Brian Sanders

New Member
I have several accounts that were affected by this migration. While I appreciate the intent to keep everyone calm, the truth is that the migration has not been as smooth at advertised. You guys need to be a little more honest about the risks involved, especially on this board and in the notifications that go out to customers. You also need to make sure that your support engineers are briefed and ready to fix migration issues when they occur. I'm working through a nasty email issue right now with KH support, and the engineer has told me that he does not get any information from the migration team.

The email for this migration was so deliberately breezy in tone, that it (rightly) set off alarm bells at my client companies who host with KH. These clients, who have had systems running smoothly on KH VPNs for years, now suddenly find themselves with email routing issues, cron issues, etc. Next time, dial back on the smoke-blowing, be honest, and make sure all your people are on the same page and ready to handle possible issues BEFORE you start migrating accounts.
 

KH-Jonathan

Director of Managed Services
Staff member
@Brian Sanders,

I'm sorry to hear that you've faced issues. I do believe from what you're saying that the issues are likely unrelated unless it's a specific MD->NJ connectivity issue: The migration process went very smoothly, and by email issue I can only assume you are referring to email going from a VPS in MD to a VPS in NJ? This was an unavoidable "problem" that only affected a few customers and was easily fixable with a static route we could put in place - and our tech. support team was fully aware of this.

Can you please let me know what ticket number you've currently got open so that I may check into what's going on for you?
 

Brian Sanders

New Member
The email issue was indeed related to the static route work-around. In this case, one server was hosting a collaboration site for a domain on another KN server. Email notifications were bouncing because they could not reach the other server. We received over 50 bounces yesterday evening alone. I don't know why you put "problem" in quotes, but for those of us dealing with actual business clients, this is most assuredly a PROBLEM, one which we were explicitly told we wouldn't notice per the migration announcement:

"By now you're probably asking how this is going to impact you, as a customer. The simple answer is you won't notice a thing. We will be live migrating our customers to the equipment located in our facility in Baltimore, MD. In performing the migration this way you will not experience any downtime or service interruption. There will be no IP addresses to change, and your entire file system with all configuration files, settings and websites will be copied over without any risk of data loss."

Here's another email-related issue that only appeared since the migration: #417492

According to one of my tickets with Henry N, a KH support engineer, "The datacenter migration is handled by our system administrators so exact details not provided on our end." That sounds to me as though the support engineers were not briefed sufficiently on the possible issues they might need to address during and after the migration.

The simple fact is that this type of migration always has the potential to disrupt service. A least be honest about those risks to customers, and don't simple cross your fingers and hope everything goes 100% smoothly. I'd be willing to bet additional issues will come to light in the coming days and weeks.
 

KH-Jonathan

Director of Managed Services
Staff member
@Brian Sanders,

Very few of our customers are using email setups between multiple East Coast VPSs, in fact I could count on one hand the number of static routes we had to setup to work around this problem. When I put problem in quotes I wasn't meaning to take away from the severity of the problem and what it could absolutely cause for business email, but I put it in quotes due to the fact that it's a problem yes, it's also by design in the way the network tunnel worked.

I just looked at ticket 417492 for you. This should be a non-issue at this point due to the tunnel no longer being active and static routes no longer being required. To add on to what was said in the ticket, it was indeed some bounces that were queued up prior to the static route being put in place for your VPS which is why you continued to receive the erroneous bounces even after the static route was setup.

The quote from Henry I believe was interpreted wrong or written poorly to reflect was was the intended statement. He was commenting on the exact migration status. He did later in that sentence explain the static route part which shows he clearly understood what the issue was and the correct resolution for it.

You're absolutely right. Any migration like this always has potential service interruption. We didn't attempt to hide any of the potential risks, but listing everything that could possibly go wrong wouldn't have served any purpose and only scared folks. There was no "crossing our fingers" involved - the migration was performed based on years of experience of our staff and administrators having done similar migrations in the past. There will almost always be a bump here or there and there were a few, but for 99% of customers they never even knew they got migrated which is exactly what we wanted. I'm terribly sorry that this did in fact impact you, however there shouldn't be any issues moving forward since the migration is done, the tunnel is no longer active, and the traffic to our east coast datacenter is now going directly to MD and not through NJ first - thus the static routes are no longer necessary at all.

I hope I've addressed all of your concerns fully. If you need anything else or have any other questions feel free to reply here or open a ticket with our support desk with Attn: Jonathan in the subject or request that it be escalated to me.
 
Top