Reclaiming Thin Provisioned Storage on Windows Using Sdelete

Many storage arrays such as HP’s 3Par line, Datacore, and others have the ability to thin provision storage.  VMware also has the ability to thin provision virtual disks (VMDK files).  Over time the storage utilization increases as data is created, fragmented, and moved.  When data is deleted the storage array does not automatically free up the disk space in the storage pool.  Block based storage arrays don’t have knowledge of what is using the LUN.  It could be a raw device or have a file system such as NTFS, VMFS, EXT3, VXFS, etc.  In order for most block based storage arrays to reclaim thin provisioned storage, the storage array has to find a zero bit pattern in an allocation unit of storage.  That way the array knows that removing an allocation unit from a LUN and returning for reuse in the storage pool does not result in lost data.  Datacore by default divides up it’s storage into 128MB segments called Storage Allocation Units or SAUs for short.  3Par uses a unit called a Chunklet which is 256MB.  If the array detects only zeros in an allocation unit, it will release it from the volume to which it is assigned and return it to the storage pool for reuse.
On Microsoft Windows the commonly accepted method to write a zero bit pattern to the free space is by using the Sysinternals utility called sdelete.  However, a change was made to the command line switches that effects the behavior.  With sdelete v1.51 the “-c” switch wrote a zero bit pattern to free space, and the “-z” switch wrote a series of ones and zeros to securely overwrite the free space.  With sdelete version 1.6 the functionality has been reversed.
If you are trying to reclaim thin provisioned space with sdelete version 1.51 or earlier you need to use the “-c” switch.  For version 1.6 you need to use the “-z” switch.  In case it changes in the future, you are looking for the switch referenced in the sdelete command line help that reads “Zero free space (good for virtual disk optimization)”.

WINDOWS - CYGWIN - Backup with RSYNC one drive to another, clone drive contents with rsync

Note this is not really a clone, clones work more with images in my opinion. They work directly on the block level instead of the file level.

I give you this which my own personal backup strategy for my external drives. I dont mind if you know this, I changed some of the words in my directories. Big woop.

One thing I dont cover is how I setup my scheduled tasks with Windows Task Scheduler, because I hate describing click by click you can google your way to how to set it up. I just used the default options that made sense and I run this script a couple times a week on Tuesday and Sunday.

4 Sections here:

1. This intro so this is the end of section 1

2. How to setup the main backup scripts, the actual meat of the backup job

3. OPTIONAL watch progress

4. OPTIONAL setup a progress script for the very first job (considering that the target was first empty, either way measures speed good)

 .....

 

Original text is HERE

How do I use Robocopy to copy ACLs without copying data?

ROBOCOPY can be used to copy the ACLs of existing files, without copying the files, but the documentation is NOT obvious on how to accomplish this.

To copy the security information for files that exits at both the source and destination, and to NOT copy the files, use:

robocopy [Source] [Destination] /secfix /xo /xn /xc

To mirror a [Source] folder on a [Destination] folder:

robocopy [Source] [Destination] /secfix /xo /xn /xc \[Other parameters\]
robocopy [Source] [Destination] /secfix \[Other parameters\]

Backup and Restore NTFS permissions with icacls

Icacls is a simple command line utility to backup and restore or apply new NTFS permissions. I use this tool mostly to back-up NTFS permissions before I make major changes on the current NTFS ACLs. This command line utility is available on Server 2003 SP2 or higher, also available on server 2008 & Windows 7.
To get help or get examples just type icacls in command prompt.
Example: Backup NTFS permissions

icacls “D:\HomeTest” /save “c:\Temp\ntfsbackup.txt” /t /c

In this example we backup all permissions of “D:\HomeTest” and save them in “c:\Temp”. The /T switch allows it to get also the permissions of sub folders, the /C switch allows it to continue even if errors occurs.
Example: Restore NTFS permissions

icacls d:/ /restore c:\Temp\ntfsbackup.txt

It is not necessary to mention the destination folder because this is already included in the backup file which very important to know. It’s sufficient to specify the destination drive.
From my experience it’s possible to back-up a complete drive like “e:/” but you can’t restore without specifying a drive letter, so my advice is always specify a folder instead of a complete drive.
More info: Technet

How To Backup ESXi Configuration

Backing up your ESXi Configuration:

 To backup your ESXi configuration you’ll be using the vicfg-cfgbackup.pl command as follows:

• Download either the vMA or vCLI
• Launch vicfg-cfgbackup.pl:
  
  C:\Program Files\VMware\VMware vSphere CLI\bin>vicfg-cfgbackup.pl --server -s name_of_the_server.bak
  
  
• Note: The backup will be stored at path:
  C:\Program Files (x86)\VMware\VMware vSphere CLI

Restoring your ESXi Configuration:

Restoring your ESXi config can be done after you have the host up and responding over the network again by using the following:

C:\Program Files\VMware\VMware vSphere CLI\bin>vicfg-cfgbackup.pl --server -l name_of_the_server.bak

Note: You will be asked to reboot the host on restore.
Backing  up multiple hosts! – There is a script to backup multiple ESXi hosts on the VMware communities site here. Also in PowerCLI here!

cfg2html on Solaris - OS configuration Backup

cfg2html is very use full script to take all the system configuration backup in text format and html format.This script is available for Solaris,various Linux flavors and HP-Unix.
For more information about the script,please visit
http://groups.yahoo.com/group/cfg2html.

Once you run the script by default it will generate three files.
1. System configuration in text format
2. System configuration in html format
3. Script Error log

These configuration backup files are very useful to build the server from scratch.But we have to  make sure you have latest configuration backup by running cfg2hmtl periodically and keep the output in other location or web portal for future reference.

Here is the script which you can download it and use it for Solaris 10.
Download cfg2html  Click on File tab- > Select Download


Download the cfg2html and keep in /var/tmp/

#cd /var/tmp
#tar -xvf cfg2html_solaris10_v1.0.tar
#cd cfg2html_solaris_10v1.0
bash-3.00# ls -lrt
total 56
-rwx------1 root root 24796 Jul 18 14:46 cfg2html_solaris_10v1.0
drwx------2 root root 11 Jul 18 14:46 plugins

bash-3.00# ./cfg2html_solaris_10v1.0
-------------------------------------------------
Starting          cfg2html_solaris_10 version 1.0 on a SunOS 5.10 i86pc
Path to cfg2html  ./cfg2html_solaris_10v1.0
Path to plugins   ./plugins
HTML Output File  /Desktop/cfg2html_solaris10_v1.0/cfg2html_solaris10_v1.0/sfos_cfg.html
Text Output File  /Desktop/cfg2html_solaris10_v1.0/cfg2html_solaris10_v1.0/sfos_cfg.txt
Errors logged to  /Desktop/cfg2html_solaris10_v1.0/cfg2html_solaris10_v1.0/sfos_cfg.err
Started at        2012-07-18 14:46:51
-------------------------------------------------
-------------------------------------------------
Collecting:  System Hardware and Operating System Summary  ..
Collecting:  Disk Device Listing  ....
Collecting:  Host-Bus Adapters (HBAs)  ...
Collecting:  Solaris Volume Manager (SVM)  ......
Collecting:  Local File Systems and Swap  ......
Collecting:  NFS Configuration  .....
Collecting:  Zone/Container Information  ......
Collecting:  Network Settings  ..........
Collecting:  EEPROM  ....
Collecting:  Cron  ...
Collecting:  System Log  ..
Collecting:  Resource Limits  .....
Collecting:  Services  ....
Collecting:  VxVM  ...........
Collecting:  VxFS  ..
-------------------------------------------------

bash-3.00# ls -lrt
total 337
-rwx------1 root root  24796 Jul 18 14:46 cfg2html_solaris_10v1.0
drwx------2 root root          11 Jul 18 14:46 plugins
-rw-r--r--   1 root     root   732 Jul 18 14:47 sfos_cfg.err
-rw-r--r--   1 root     root   77295 Jul 18 14:47 sfos_cfg.html
-rw-r--r--   1 root     root   65492 Jul 18 14:47 sfos_cfg.txt
bash-3.00# uname -a
SunOS sfos 5.10 Generic_142910-17 i86pc i386 i86pc

You can copy to windows and upload it in desired location .

you can add the cfg2hmtl  in cronjob to run cfg2html periodically .

# export EDITOR=vi
# corntab -e
Add the below lines in the end of the file.
00 23 15 * * /var/tmp/cfg2html_solaris10_v1.0/cfg2html_solaris_10v1.0  > /dev/null 2> /dev/null
00 23 01 * * /var/tmp/cfg2html_solaris10_v1.0/cfg2html_solaris_10v1.0  > /dev/null 2> /dev/null

save the file & exit.The above job will run cfg2html 1st and 15th of the month at 11PM .

cfg2html on Solaris - OS configuration Backup

at 15:22

Lingeswaran R

Solaris 10

---

cfg2html is very use full script to take all the system configuration backup in text format and html format.This script is available for Solaris,various Linux flavors and HP-Unix.
For more information about the script,please visit 
http://groups.yahoo.com/group/cfg2html.

Once you run the script by default it will generate three files. 
1. System configuration in text format  
2. System configuration in html format
3. Script Error log

These configuration backup files are very useful to build the server from scratch.But we have to  make sure you have latest configuration backup by running cfg2hmtl periodically and keep the output in other location or web portal for future reference.

Here is the script which you can download it and use it for Solaris 10.
Download cfg2html  Click on File tab- > Select Download 


Download the cfg2html and keep in /var/tmp/

#cd /var/tmp
#tar -xvf cfg2html_solaris10_v1.0.tar
#cd cfg2html_solaris_10v1.0
bash-3.00# ls -lrt
total 56
-rwx------1 root root 24796 Jul 18 14:46 cfg2html_solaris_10v1.0
drwx------2 root root 11 Jul 18 14:46 plugins

bash-3.00# ./cfg2html_solaris_10v1.0
-------------------------------------------------
Starting          cfg2html_solaris_10 version 1.0 on a SunOS 5.10 i86pc
Path to cfg2html  ./cfg2html_solaris_10v1.0
Path to plugins   ./plugins
HTML Output File  /Desktop/cfg2html_solaris10_v1.0/cfg2html_solaris10_v1.0/sfos_cfg.html
Text Output File  /Desktop/cfg2html_solaris10_v1.0/cfg2html_solaris10_v1.0/sfos_cfg.txt
Errors logged to  /Desktop/cfg2html_solaris10_v1.0/cfg2html_solaris10_v1.0/sfos_cfg.err
Started at        2012-07-18 14:46:51
-------------------------------------------------
-------------------------------------------------
Collecting:  System Hardware and Operating System Summary  ..
Collecting:  Disk Device Listing  ....
Collecting:  Host-Bus Adapters (HBAs)  ...
Collecting:  Solaris Volume Manager (SVM)  ......
Collecting:  Local File Systems and Swap  ......
Collecting:  NFS Configuration  .....
Collecting:  Zone/Container Information  ......
Collecting:  Network Settings  ..........
Collecting:  EEPROM  ....
Collecting:  Cron  ...
Collecting:  System Log  ..
Collecting:  Resource Limits  .....
Collecting:  Services  ....
Collecting:  VxVM  ...........
Collecting:  VxFS  ..
-------------------------------------------------

bash-3.00# ls -lrt
total 337
-rwx------1 root root  24796 Jul 18 14:46 cfg2html_solaris_10v1.0
drwx------2 root root          11 Jul 18 14:46 plugins
-rw-r--r--   1 root     root   732 Jul 18 14:47 sfos_cfg.err
-rw-r--r--   1 root     root   77295 Jul 18 14:47 sfos_cfg.html
-rw-r--r--   1 root     root   65492 Jul 18 14:47 sfos_cfg.txt
bash-3.00# uname -a
SunOS sfos 5.10 Generic_142910-17 i86pc i386 i86pc

You can copy to windows and upload it in desired location .

you can add the cfg2hmtl  in cronjob to run cfg2html periodically .

# export EDITOR=vi
# corntab -e
Add the below lines in the end of the file.
00 23 15 * * /var/tmp/cfg2html_solaris10_v1.0/cfg2html_solaris_10v1.0  > /dev/null 2> /dev/null
00 23 01 * * /var/tmp/cfg2html_solaris10_v1.0/cfg2html_solaris_10v1.0  > /dev/null 2> /dev/null

save the file & exit.The above job will run cfg2html 1st and 15th of the month at 11PM .

- See more at: http://www.unixarena.com/2012/07/cfg2html-on-solaris-os-configuration.html#sthash.jbCYLK2o.dpuf

HOW-TO get MP IP address from HP-UX

I found out that now there is a posibility to get the IP of the MP/GSP from the HP-UX servers, just in case you forgot or can't phisicaly access the server:

You need to have installed the sfm product:

SFM-CORE B.05.00.05 HPUX System Fault Management

It commes standard with the 11.23 FOE, so probably you have it by default.

Then you just lauch:

 #/opt/sfm/bin/CIMUtil -e root/cimv2 HP_ManagementProcessor  
 Instance 0 :  
 UniqueIdentifier : 0.19.33.124.117.193  
 ControllerType : 3  
 OtherControllerType :  
 IPAddress : 10.10.11.250 <------ MP IP  
 URL : http://10.10.11.250 <------ MP IP  
 Dedicated : 14  
 CreationClassName : HP_ManagementProcessor  
 Name : Management Processor  
 EnabledState : 2  
 OperationalStatus : 2  

I found out that now there is a posibility to get the IP of the MP/GSP from the HP-UX servers, just in case you forgot or can't phisicaly access the server:
You need to have installed the sfm product:
SFM-CORE B.05.00.05 HPUX System Fault Management
It commes standard with the 11.23 FOE, so probably you have it by default.
Then you just lauch:
#/opt/sfm/bin/CIMUtil -e root/cimv2 HP_ManagementProcessor
Instance 0 :
UniqueIdentifier : 0.19.33.124.117.193
ControllerType : 3
OtherControllerType :
IPAddress : 10.10.11.250 <------ br="" ip="" mp=""> URL : http://10.10.11.250 <------ br="" ip="" mp=""> Dedicated : 14
CreationClassName : HP_ManagementProcessor
Name : Management Processor
EnabledState : 2
OperationalStatus : 2
- See more at: http://www.hpuxtips.es/?q=node/175#sthash.r8RDvWar.dpuf

I found out that now there is a posibility to get the IP of the MP/GSP from the HP-UX servers, just in case you forgot or can't phisicaly access the server:
You need to have installed the sfm product:
SFM-CORE B.05.00.05 HPUX System Fault Management
It commes standard with the 11.23 FOE, so probably you have it by default.
Then you just lauch:
#/opt/sfm/bin/CIMUtil -e root/cimv2 HP_ManagementProcessor
Instance 0 :
UniqueIdentifier : 0.19.33.124.117.193
ControllerType : 3
OtherControllerType :
IPAddress : 10.10.11.250 <------ br="" ip="" mp=""> URL : http://10.10.11.250 <------ br="" ip="" mp=""> Dedicated : 14
CreationClassName : HP_ManagementProcessor
Name : Management Processor
EnabledState : 2
OperationalStatus : 2
- See more at: http://www.hpuxtips.es/?q=node/175#sthash.r8RDvWar.dpuf