Monday, September 16, 2013

Get quick, free stats from your vSphere environment with RVTools

RVTools is a FREE Windows .NET 2.0 application which uses the VI SDK to display information about your virtual machines and ESX/ESXi hosts. RVTools works with:
  • VirtualCenter/vCenter 2.5 and higher (including vCenter 5.1)
  • ESX/ESXi/vSphere 3.5 and higher (including vSphere 5.1)
RVTools is able to list information about:
  • Virtual machines
  • CPU
  • Memory
  • Disks
  • Partitions
  • Network
  • Floppy drives
  • CD/DVD drives
  • Snapshots
  • VMware tools
  • Resource pools
  • ESX hosts
  • HBAs
  • NICs
  • Switches
  • Ports
  • Distributed Switches
  • Distributed Ports
  • Service consoles
  • VM Kernels
  • Datastores
With RVTools you can disconnect the CD-ROM or floppy drives from the virtual machines and RVTools is able to update the VMware Tools installed inside each virtual machine to the latest version.

Sunday, September 15, 2013

3 Command line tool to test bandwidth between 2 servers

One element that is often not know, or that should be measured after a problem statement or after a change in the infrastructure is the network . But how do you accurately measure the speed between two servers?
Someone use ftp, scp or other file transfer protocols, these can give some indication, but probably you’ll measure the limit of your disks or CPU.
In this article I will show you 3 way to measure the bandwidth from the command line, without using the disks.



Iperf

Iperf was developed by NLANR/DAST as a modern alternative for measuring maximum TCP and UDP bandwidth performance. Iperf allows the tuning of various parameters and UDP characteristics. Iperf reports bandwidth, delay jitter, datagram loss.
The quality of a link can be tested as follows:
- Latency (response time or RTT): can be measured with the Ping command.
- Jitter (latency variation): can be measured with an Iperf UDP test.
- Datagram loss: can be measured with an Iperf UDP test.
The bandwidth is measured through TCP tests.
To be clear, the difference between TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) is that TCP use processes to check that the packets are correctly sent to the receiver whereas with UDP the packets are sent without any checks but with the advantage of being quicker than TCP.
Iperf uses the different capacities of TCP and UDP to provide statistics about network links.
With Iperf you have a server machine where iperf put itself in listening and the other that is the client that send the informations.
Example:
iperf
Basic usage:
Server side:

#iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[852] local 10.1.1.1 port 5001 connected with 10.6.2.5 port 54355
[ ID]   Interval          Transfer        Bandwidth
[852]   0.0-10.1 sec   1.15 MBytes   956 Kbits/sec
------------------------------------------------------------
Client connecting to 10.6.2.5, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[824] local 10.1.1.1 port 1646 connected with 10.6.2.5 port 5001
[ ID]   Interval          Transfer        Bandwidth
[824]   0.0-10.0 sec   73.3 MBytes   61.4 Mbits/sec
Client side

#iperf -c 10.1.1.1 -d
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 10.1.1.1, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 5] local 10.6.2.5 port 60270 connected with 10.1.1.1 port 5001
[ 4] local 10.6.2.5 port 5001 connected with 10.1.1.1 port 2643
[ 4] 0.0-10.0 sec 76.3 MBytes 63.9 Mbits/sec
[ 5] 0.0-10.1 sec 1.55 MBytes 1.29 Mbits/sec
So using Iperf (with appropriate flags) on both our machines we can simply measure the bandwidth between them.
Iperf is available also for Windows.
Complete guide: http://openmaniak.com/iperf.php

Netcat

To eliminate the disks from having any part of the transfer, we will use netcat transferring the output of command yes. Netcat is described as being a “feature-rich network debugging and exploration tool”. It can be obtained from Source Forge, or it may already be available in your distribution.
Again we will use one of the machines as a server that receives the data and the other as a client that sends the information.
Basic usage
On th server machine
nc -v -v -l -n  2222 >/dev/null
listening on [any] 2222 ...
On the client machine
time yes|nc -v -v -n 10.1.1.1 2222 >/dev/null
On client stop the process  after 10 seconds (more or less) with ctrl-c, you’ll get something like:
sent 87478272, rcvd 0
 
real 0m9.993s
user 0m2.075s
sys 0m0.939s
On the server machine, note the data received (in bytes)
 sent 0, rcvd 87478392
Now multiply the bytes rcvd by 8 to get total bits, then divide by the time: Result in this example is 70Mb/s
Reference: http://deice.daug.net/netcat_speed.html

Bandwidth Test Controller (BWCTL)

BWCTL is a command line client application and a scheduling and policy daemon. These tests can measure maximum TCP bandwidth, with various tuning options available, or, by doing a UDP test, the delay, jitter, and datagram loss of a network.
The bwctl client application works by contacting a bwctld process on the two test endpoint systems. BWCTL will work as a 3-party application. The client can arrange a test between two servers on two different systems. If the local system is intended to be one of the endpoints of the test, bwctl will detect whether a local bwctld is running and will handle the required server functionality if needed.
The bwctl client is used to request the type of throughput test wanted. Furthermore, it requests when the test should be executed. bwctld either responds with a tentative reservation or a test denied message. Once bwctl is able to get a matching reservation from both bwctld processes (one for each host involved in the test), it confirms the reservation. Then, the bwctld processes run the test and return the results. The results are returned to the client from both sides of the test. Additionally, the bwctld processes share the results from their respective sides of the test with each other.
bwctl_arch
For more information check the man page: http://www.internet2.edu/performance/bwctl/manpages.html

Friday, September 13, 2013

Reclaiming Thin Provisioned Storage on Windows Using Sdelete

Many storage arrays such as HP’s 3Par line, Datacore, and others have the ability to thin provision storage.  VMware also has the ability to thin provision virtual disks (VMDK files).  Over time the storage utilization increases as data is created, fragmented, and moved.  When data is deleted the storage array does not automatically free up the disk space in the storage pool.  Block based storage arrays don’t have knowledge of what is using the LUN.  It could be a raw device or have a file system such as NTFS, VMFS, EXT3, VXFS, etc.  In order for most block based storage arrays to reclaim thin provisioned storage, the storage array has to find a zero bit pattern in an allocation unit of storage.  That way the array knows that removing an allocation unit from a LUN and returning for reuse in the storage pool does not result in lost data.  Datacore by default divides up it’s storage into 128MB segments called Storage Allocation Units or SAUs for short.  3Par uses a unit called a Chunklet which is 256MB.  If the array detects only zeros in an allocation unit, it will release it from the volume to which it is assigned and return it to the storage pool for reuse.
On Microsoft Windows the commonly accepted method to write a zero bit pattern to the free space is by using the Sysinternals utility called sdelete.  However, a change was made to the command line switches that effects the behavior.  With sdelete v1.51 the “-c” switch wrote a zero bit pattern to free space, and the “-z” switch wrote a series of ones and zeros to securely overwrite the free space.  With sdelete version 1.6 the functionality has been reversed.
If you are trying to reclaim thin provisioned space with sdelete version 1.51 or earlier you need to use the “-c” switchFor version 1.6 you need to use the “-z” switch.  In case it changes in the future, you are looking for the switch referenced in the sdelete command line help that reads “Zero free space (good for virtual disk optimization)”.

Monday, September 9, 2013

WINDOWS - CYGWIN - Backup with RSYNC one drive to another, clone drive contents with rsync

Note this is not really a clone, clones work more with images in my opinion. They work directly on the block level instead of the file level.

I give you this which my own personal backup strategy for my external drives. I dont mind if you know this, I changed some of the words in my directories. Big woop.

One thing I dont cover is how I setup my scheduled tasks with Windows Task Scheduler, because I hate describing click by click you can google your way to how to set it up. I just used the default options that made sense and I run this script a couple times a week on Tuesday and Sunday.

4 Sections here:
1. This intro so this is the end of section 1
2. How to setup the main backup scripts, the actual meat of the backup job
3. OPTIONAL watch progress
4. OPTIONAL setup a progress script for the very first job (considering that the target was first empty, either way measures speed good)
 .....
 
Original text is HERE

How do I use Robocopy to copy ACLs without copying data?

ROBOCOPY can be used to copy the ACLs of existing files, without copying the files, but the documentation is NOT obvious on how to accomplish this.

To copy the security information for files that exits at both the source and destination, and to NOT copy the files, use:

robocopy [Source] [Destination] /secfix /xo /xn /xc

To mirror a [Source] folder on a [Destination] folder:

robocopy [Source] [Destination] /secfix /xo /xn /xc \[Other parameters\]
robocopy [Source] [Destination] /secfix \[Other parameters\]

Backup and Restore NTFS permissions with icacls

Icacls is a simple command line utility to backup and restore or apply new NTFS permissions. I use this tool mostly to back-up NTFS permissions before I make major changes on the current NTFS ACLs. This command line utility is available on Server 2003 SP2 or higher, also available on server 2008 & Windows 7.
To get help or get examples just type icacls in command prompt.
Example: Backup NTFS permissions
icacls “D:\HomeTest” /save “c:\Temp\ntfsbackup.txt” /t /c

In this example we backup all permissions of “D:\HomeTest” and save them in “c:\Temp”. The /T switch allows it to get also the permissions of sub folders, the /C switch allows it to continue even if errors occurs.
Example: Restore NTFS permissions
icacls d:/ /restore c:\Temp\ntfsbackup.txt

It is not necessary to mention the destination folder because this is already included in the backup file which very important to know. It’s sufficient to specify the destination drive.
From my experience it’s possible to back-up a complete drive like “e:/” but you can’t restore without specifying a drive letter, so my advice is always specify a folder instead of a complete drive.
More info: Technet

How to use DiskSpd to simulate Veeam Backup & Replication disk actions

This HOW-TO contains information on how to use Microsoft© DiskSpd to simulate Veeam Backup & Replication disk actions to measure disk pe...