Hi,
We had an old vmware server on a 2008 server, this has 3 virtual machines. One I installed and two that were installed before i was here.
We converted these a few weeks ago to esxi 5.0.
Since then we seem to have some issues on two of the three ubuntu vm's (the one I installed works correctly).
Network is up and i have a constant ping open from and to machine.207 (one that has issues).This is the issue: unstable tcp connections. When you try to ssh to it it just drops the connection, second time does work sometimes. When working in the ssh session you get disconnected randomly.
What I have done so far:
- upgraded all ubuntu's to latest 12.04 LTS
- disabled ipv6
- checked routing on .207 -> just has a default route to our firewall, routing is not the issue as this happens locally.
- checked arp = ok
- rebooted the machines -> issue is the same
Now I installed wireshark and this is what happens when it fails:
i see from my machine (.100) to .207 -> 335 19.989077000 192.168.0.100 192.168.0.207 TCP 54 59763 > ssh [ACK] Seq=1 Ack=1 Win=65536 Len=0
.207 answers back -> 335 19.989077000 192.168.0.100 192.168.0.207 TCP 54 59763 > ssh [ACK] Seq=1 Ack=1 Win=65536 Len=0
368 21.389081000 192.168.0.207 192.168.0.100 TCP 66 ssh > 59763 [SYN, ACK] Seq=0 Ack=1 Win=14600 Len=0 MSS=1460 SACK_PERM=1 WS=16
555 23.389142000 192.168.0.207 192.168.0.100 TCP 66 ssh > 59763 [SYN, ACK] Seq=0 Ack=1 Win=14600 Len=0 MSS=1460 SACK_PERM=1 WS=16
765 27.389213000 192.168.0.207 192.168.0.100 TCP 66 ssh > 59763 [SYN, ACK] Seq=0 Ack=1 Win=14600 Len=0 MSS=1460 SACK_PERM=1 WS=16
970 35.389339000 192.168.0.207 192.168.0.100 TCP 66 ssh > 59763 [SYN, ACK] Seq=0 Ack=1 Win=14600 Len=0 MSS=1460 SACK_PERM=1 WS=16
1307 51.389592000 192.168.0.207 192.168.0.100 TCP 66 ssh > 59763 [SYN, ACK] Seq=0 Ack=1 Win=14600 Len=0 MSS=1460 SACK_PERM=1 WS=16
1417 58.524460000 192.168.0.207 192.168.0.100 TCP 60 ssh > 58009 [ACK] Seq=1 Ack=1 Win=2532 Len=0
3777 133.529204000 192.168.0.207 192.168.0.100 TCP 60 [TCP Dup ACK 1417#1] ssh > 58009 [ACK] Seq=1 Ack=1 Win=2532 Len=0
8144 208.534869000 192.168.0.207 192.168.0.100 TCP 60 [TCP Dup ACK 1417#2] ssh > 58009 [ACK] Seq=1 Ack=1 Win=2532 Len=0
12679 283.539682000 192.168.0.207 192.168.0.100 TCP 60 ssh > 58009 [RST, ACK] Seq=2 Ack=1 Win=2532 Len=0
I have no idea why i get a reset back. I'm in the dark 
iptables -L on .207 gives nothing.
This is a successfull attempt, then dropping after a while:
48 2.635152000 192.168.0.207 192.168.0.100 TCP 66 ssh > 59999 [SYN, ACK] Seq=0 Ack=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 WS=16
76 3.911437000 192.168.0.100 192.168.0.207 TCP 66 60000 > ssh [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1
77 3.911732000 192.168.0.207 192.168.0.100 TCP 66 ssh > 60000 [SYN, ACK] Seq=0 Ack=1 Win=14600 Len=0 MSS=1460 SACK_PERM=1 WS=16
78 3.911834000 192.168.0.100 192.168.0.207 TCP 54 60000 > ssh [ACK] Seq=1 Ack=1 Win=65536 Len=0
then i wait a while, i see some traffic and then this happens:
2165 113.070976000 192.168.0.207 192.168.0.100 SSHv2 106 Encrypted response packet len=52
2166 113.125807000 192.168.0.100 192.168.0.207 TCP 54 60000 > ssh [ACK] Seq=1917 Ack=3640 Win=65536 Len=0
2168 113.126155000 192.168.0.207 192.168.0.100 TCP 60 ssh > 60000 [RST] Seq=3640 Win=0 Len=0
2169 113.288979000 192.168.0.207 192.168.0.100 SSHv2 106 [TCP Retransmission] Encrypted response packet len=52
repeated a few times
and ends with:
4660 225.774711000 192.168.0.207 192.168.0.100 SSHv2 106 [TCP Retransmission] Encrypted response packet len=52
what has been tried also so far:
switch network card in vm from e1000 to vmxnet3
no iptables is active on the system
tried installing telnet deamon -> has same issues as the ssh connection, ftp connection does weird also
174601 699.510273000 192.168.0.207 192.168.0.100 SSHv2 106 [TCP Retransmission] Encrypted response packet len=52
296330 819.832237000 192.168.0.207 192.168.0.100 SSHv2 106 [TCP Retransmission] Encrypted response packet len=52
if more will be logged beside this one, I'll update this post.