Tuesday, November 17, 2020

How to use DiskSpd to simulate Veeam Backup & Replication disk actions

This HOW-TO contains information on how to use Microsoft© DiskSpd to simulate Veeam Backup & Replication disk actions to measure disk performance.

The test file created by DiskSpd does not contain any diagnostic information and must be removed manually after testing has concluded. All diagnostic information regarding the performance test is displayed in the command line.

Active full or forward incremental

C:\diskspd> diskspd.exe -c25G -b512K -w100 -Sh -d600 D:\testfile.dat

-w100 indicates 100% writes and 0% reads. Sequential I/O is used by default.

Reverse incremental

C:\diskspd> diskspd.exe -c100G -b512K -w67 -r4K -Sh -d600 D:\testfile.dat

-w67 indicates 67% writes and 33% reads to simulate 2 write and 1 read operations that happen in reverse incremental backup jobs.

-r4K enables random I/O that are 4KB aligned, for a more realistic simulation.

After the test has finished, take Total IO MB/s from the results and divide it by 3. This is because for every processed block Veeam needs to do 3 I/O operations, thus the effective speed is 3 times slower.

Transforms, merges, and other synthetic operations

Includes transformation of incrementals to rollbacks, merge operations in forever forward incremental backups and backup copy jobs, and creation of synthetic full backup files and GFS points.

C:\diskspd\x86fre> diskspd.exe -c100G -b512K -w50 -r4K -Sh -d600 D:\testfile.dat

-w50 indicates 50% writes and 50% reads to simulate reading data from one file and writing that data into another (or in the case of transform, reading the same number of blocks from two files as are written to two other files).

-r4K enables random I/O that are 4KB aligned, for a more realistic simulation.

After the test has finished, take Total IO MB/s from the results and divide it by 2 (4 for transform to rollbacks). This is because for every processed block Veeam needs to do 2 I/O operations, thus the effective speed is 2 times slower. For transform to rollbacks, each block must be read out of the full backup file and written into the rollback before the corresponding block can be read out of the incremental and written into the full, which results in 4x I/O.

To estimate an expected time to complete the synthetic operation, in seconds:

For synthetic full backup and GFS points: divide the expected size of the new full backup file (typically the same as previous full backup files) by the effective speed.

For all other synthetic operations, add the sizes of all of the incremental files which will be merged or transformed, and then divide the resulting sum by the effective speed. Typically only the oldest incremental file is merged, whereas all incremental files are transformed to rollbacks.

Slow restore or Surebackup

This is typically when you're restoring from deduplication appliances with sub-optimal settings. As a workaround in case of slow restore, manually copy the backup files elsewhere (e.g. Veeam server), import and restore from there.

Worst case scenario where the backup file is heavily fragmented inside, which implies a lot of random read I/O:

C:\diskspd\x86fre> diskspd.exe -b512K -r4K -Sh -d600 \\nas\share\VeeamBackups\Job\Job2014-01-23T012345.vbk

-r4K enables random I/O that are 4KB aligned, for a more realistic simulation.

Best case scenario where the backup file is not fragmented inside (no parallel processing), which implies linear read I/O: 

C:\diskspd\x86fre> diskspd.exe -b512K -Sh -d600 \\nas\share\VeeamBackups\Job\Job2014-01-23T012345.vbk

In both cases you need to pick an existing .vbk file as the target. Only read operations will be performed.

Direct disk access speed

C:\diskspd\x86fre> diskspd.exe -Sh -d600 #X

Where X is the number of disk that you see in Disk Management.

This test will not overwrite any data, it is a safe test, and it works for Offline disks too. You can simulate and measure maximum possible reading speed in SAN or hot-add modes, however this of course will not take any VDDK overhead into account.

Tuesday, October 6, 2020

Changing Qlogic Queue Depth Windows

 The maximum SCSI queue depth value, as determined by the Microsoft port driver developer, is 254. QLogic sets the default value to 32. The Microsoft storport.sys driver has total control of the depth of the device queue; QLogic Miniport merely sets the value using the predefined Storport interface call and procedure as determined by Microsoft.

 
Use the following procedure to change the queue depth parameter:
 
1. Click

Start, select Run, and open the REGEDIT/REGEDT32 program.

2. Select

HKEY_LOCAL_MACHINE and follow the tree structure down to the QLogic driver as follows:
HKEY_LOCAL_MACHINE
SYSTEM
CurrentControlSet
Services
ql2300
Parameters
Device

3. Double-click DriverParameter to edit the string, for example:

DriverParameter REG_SZ qd=32

4. Add "qd=" to the "Value data:" field. If the string "qd=" does not exist, append it to end of the string with a semicolon, for example:

;qd=<value>

5. Enter a value up to 254 (0xFE).

The default value is 32 (0x20).

6. Click OK.
7. Exit the Registry Editor, then shutdown and reboot the system.

Thursday, September 24, 2020

Reset iLO4 certificate

 During troubleshooting of issue at my customer in OneView, I have noticed that there is a SSL certificate issue with one of iLO4 of BL460gen9 ... 

After seraching HPE support page, I have stumbled to this article:

https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=emr_na-a00042194en_us

Advisory: (Revision) HP Integrated Lights-Out (iLO) - iLO 3 and iLO 4 Self-Signed SSL Certificate May Have an Expiration Date Earlier Than the Issued Date

The HP Integrated Lights-Out 3 (iLO 3) and HP Integrated Lights-Out 4 (HP iLO 4) may have a self-signed Secure Socket Layer (SSL) certificate containing the incorrect year for the expiration date as follows:




RESOLUTION

Import a Trusted SSL Certificate signed by a Certification Authority (CA) into iLO 3 and iLO 4. Refer to the iLO User Guide for more details.

The HP iLO 4 User Guide is located at the following URL: https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c03334051 The HP iLO 3 User Guide is located at the following URL: https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c02774507

If a Certification Authority is not available, perform either of the following steps as a workaround:

Downgrade the iLO firmware

  1. Downgrade the iLO 3 firmware to Version 1.50 or the iLO 4 firmware to Version 1.13.
  2. Boot the server and press F9 to enter the ROM-Based Setup Utility (RBSU).
  3. In the RBSU Date and Time Menu, set the system date to 01-01-2013.
  4. Save changes and exit the RBSU.
  5. Wait until the server completes the Power-On Self-Test (POST).
  6. Log into iLO, verify the iLO date is January 1, 2013, and force iLO to generate a new self-signed SSL certificate by either changing the iLO hostname or restoring iLO to factory defaults.
  7. After iLO has a new self-signed SSL certificate containing a valid expiration date, the iLO 3 firmware can be upgraded to Version 1.55 and the iLO 4 firmware can be upgraded to Version 1.20.

OR

iLO 4 firmware version 2.55 (or later)

The iLO 4 self-signed certificate can also be regenerated using a REST command that was added in iLO 4 firmware 2.55 (or later). To regenerate the self-signed certificate using this REST command use the following syntax:

DELETE https://{iLO}/redfish/v1/Managers/{item}/SecurityService/HttpsCert/

Below is an example using curl.

curl -X DELETE https://ilo.example.net/redfish/v1/Managers/1/SecurityService/HttpsCert/

HTTP/1.1 200 OK
Cache-Control: no-cache
Content-length: 299
Content-type: application/json; charset=utf-8
Date: Sat, 20 Jan 2018 03:38:28 GMT
ETag: W/"78E0DBAE"
X-Frame-Options: sameorigin
X_HP-CHRP-Service-Version: 1.0.3

{"Messages":[{"MessageID":"iLO.0.10.ImportCertSuccessfuliLOResetinProgress"}],"Type":"ExtendedError.1.0.0","error":{"@Message.ExtendedInfo": [{"MessageID":"iLO.0.10.ImportCertSuccessfuliLOResetinProgress"}],"code":"iLO.0.10.ExtendedInfo","message":"See @Message.ExtendedInfo for mo re information."}}

The expiration date on the self-signed SSL certificate is targeted to be resolved in a future version of the iLO 3 and iLO 4 firmware.


So I have tried to use 2nd option, as I have 2.55+ firmware.

You will need CURL for windows, and some extra command parameters to add so that CURL delete process can work:

curl -X DELETE https://iLO_IP_ADDRESS/redfish/v1/Managers/1/SecurityService/HttpsCert/ -i --insecure -u username:password -L

Thursday, October 11, 2018

VMware ESXi SCSI Sense Code Decoder

https://www.virten.net/vmware/esxi-scsi-sense-code-decoder/

SCSI errors or warnings in ESXi are displayed with 6 status codes. This page converts these sense codes from ESXi Hosts to human readable status information. In the vmkernel.log system log file from an ESXi 5.x or 6.0 hosts, you see entries similar to the following.

ScsiDeviceIO: [...] Cmd 0x1a [...] to dev "naa.x" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.




Monday, August 20, 2018

HP ILO SSL_ERROR_BAD_MAC_ALERT Firefox Chrome

The wonderful security updates in Firefox (and Chrome) will give you the dreaded SSL_ERROR_BAD_MAC_ALERT. So no more ILO for you 
How to mitigate that?, well first do as told in the article above, go to about:config in firefox and set the security.tls.version.fallback-limit to 1.
How you can access the ILO and update the firmware to 2.29 for ILO2 (or higher), as instructed here, you can fetch the ILO2 firmware  Here
HP has a habit of only updating the Windoze firmware binaries, forgetting that most of the internet runs on Linux, but anyway just download the EXE and extract the ilo229.bin from the file.
If you don’t like the clickerdyclick web interface, or have more than 2 servers to update, it makes sense to put your binary on some web server  in your environment and just ssh into your ILO and update from there, the magic is done like so:
load -source http://192.168.1.11/iso/HP/Firmware/ilo2_229.bin /map1/firmware1
(192.168.1.11/iso…. is our internal web server of course, replace with your own)
O and don’t forget to set your FireFox security.tls.version.fallback-limit back to 3

Monday, October 23, 2017

Installing Java 7, 8, and 9 on Ubuntu and Linux Mint

original post from: http://dcjtech.info/topic/installing-java-7-8-and-9-on-ubuntu-and-linux-mint/

Java is used by many computer users. Obviously, Ubuntu and Linux Mint users need Java and they may wish to use Java 7, 8, or 9. Currently, Oracle's Java is not in the default repos, so here is how users can install it.

Java 7

Run the below code in a command-line with Root privileges.
add-apt-repository ppa:webupd8team/java

apt-get update

apt-get install oracle-java7-installer
Running "apt-get install oracle-java7-set-default" will set the global Java 7 system/environment variables and remove the variables of other Java versions.

Java 8

Run the below code in a command-line with Root privileges.
add-apt-repository ppa:webupd8team/java

apt-get update

apt-get install oracle-java8-installer
The "oracle-java8-installer" package downloads an installer that obtain Java from Oracle directly. Java itself is not in the PPA.
Running "apt-get install oracle-java8-set-default" will set the global Java 8 system/environment variables and remove the variables of other Java versions..

Java 9

Run the below code in a command-line with Root privileges to obtain Java 9 on Ubuntu, Linux Mint, and other distros that support PPAs.
add-apt-repository ppa:webupd8team/java

apt-get update

apt-get install oracle-java9-installer

Verify Installations

Java installations can be verified by executing "java -version" or "javac -version".

Switch Between Javas

To make Java 7 the default, run "update-java-alternatives -s java-7-oracle". Alternately, to make Java 8 the default, run "update-java-alternatives -s java-8-oracle". Obviously, "update-java-alternatives -s java-9-oracle" makes Java 9 the default. All of these commands require Root privileges.

Sunday, August 20, 2017

HP 3PAR REPLACING A FAILED DISK


Replacing a failed disk in a 3PAR is pretty simple you just need to follow a few steps to make sure you do it safely. If you are new to 3PAR or would like to learn more a good place to start is our 3PAR beginners guide
Let’s get started with the disk replacement procedure:
1 Check to see if you have any failed or degraded disks in the system. Take a note of the disk ID and cage position. In this case disk ID =26, cage position = 2:8:0
3PARSAN01 cli% showpd -failed -degraded

                           -Size(MB)-- ----Ports----

Id CagePos Type RPM State   Total Free A     B     Cap(GB)

46 2:8:0? FC   10 failed 417792   0 ----- -----     450

------------------------------------------------------------

 1 total                  417792   0
2 Check if the disk sevicemag command is running on the drive. The servicemag command is used to inform the system to evacuate all the chunklets from a drive so that it is ready for service. Below we can see the servicemag has succeeded on the drive we identified in step 1.
3PARSAN01 cli% servicemag status

Cage 2, magazine 8:

The magazine was successfully brought offline by a servicemag start command.

The command completed Thu Jul 10 20:07:03 2014.

servicemag start -pdid 46 – Succeeded
3 Next we double check there is no data left on the drive. You can do this by running showpd –space driveID as below. You need to check that all columns other than size and failed are zero
3PARSAN01 cli% showpd –space 46

Id CagePos Type -State-   Size Volume Spare Free Unavail Failed
46 2:8:0? FC   failed 417792     0                 0     0          0         417792

---------------------------------------------------------------

1 total                        417792     0            0   0          0        417792

4 Next to replace the physical disk. Make sure you are happy with the above steps. Then pop that bad boy out, you will have a note of the location of the failed drive from step 1.

5 Once the disk is in you can monitor the progress of the rebuild by running servicemag status, which will give you an ETA for completion.
3PARSAN01 cli% servicemag status

Cage 2, magazine 8:

The magazine is being brought online due to a servicemag resume.

The last status update was at Thu Jun 26 12:09:19 2014.

Chunklets relocated: 73 in 50 minutes and 34 seconds

Chunklets remaining: 400

Chunklets marked for moving: 400

Estimated time for relocation completion based on 41 seconds per chunklet is: 4hours, 57 minutes and 39 seconds

servicemag resume 2 8 -- is in Progress
6 Once this is complete you can check that your disks are showing in a normal state with showpd -state
3PARSAN01 cli% showpd -state

Id CagePos Type -State- --------------------Detailed_State---------------------

43 2:5:0   FC   normal normal

44 2:6:0   FC   normal normal

45 2:7:0   FC   normal normal

46 2:8:0 FC   normal normal

47 2:9:0   FC   normal normal
That’s it, job done!

How to use DiskSpd to simulate Veeam Backup & Replication disk actions

This HOW-TO contains information on how to use Microsoft© DiskSpd to simulate Veeam Backup & Replication disk actions to measure disk pe...