Storage Essentials : Beware ‘Get All Details’ Task
Shortly after the introduction of Storage Essentials into an environment, fibre tape backups were interrupted at around 03:00 AM every day. We had various errors reported by hosts:
[Major] From: [email protected] “SYS-LTO5_LIB_Drive_1” Time: 23/02/2011 03:02:56
[90:51] Tape8:0:6:0C Cannot write to device (Details unknown.)[Major] From: [email protected] “CDC-LTO5_LIB_Drive_4” Time: 23/02/2011 03:03:26
[90:161] Cannot write filemark. ([5] I/O error)[Critical] From: [email protected] “/itaedi” Time: 23/02/2011 03:03:26
Unexpected close reading NET message => aborting.[Major] From: [email protected] “UX-ITPROD_OFFLINE” Time: 23/02/2011 03:04:52
[61:3003] Lost connection to VBDA named “/itaedi”
on host server.domain.local.
Ipc subsystem reports: “IPC Read Error
System error: [10053] Software caused connection abort
“
[Minor] From: [email protected] “evamgmt02.domain.local [/CONFIGURATION]” Time: 22/02/2011 22:44:13
[81:141] \IISDatabase
Cannot export configuration object: (Unknown internal error.) => backup incomplete.
It turned out that there was a scheduled task within Storage Essentials ‘Get All Details‘ that ran at 03:00 AM every day. This task scanned all known hosts to check for new fibre attached LUNs/devices.
After disabling this task backups were no longer interrupted. It is still possible tomanually trigger updateson a single host, this does NOT impact backups that are runnin, it only appears that backups are interrupted whilst running the scan on ALL hosts.