Some iland customers are experiencing degraded performance with their cloud services in our Sterling (STR02). We are aware of this issue and are working to resolve it quickly.
Several customers have reported long running backup jobs and/or job failures with the iland Secure Cloud Backup environment. This is a result of a series of hardware and software errors that we have experienced recently. We are working with our vendors to troubleshoot these errors and expect to resolve both issues quickly.
How we are resolving it:
From a hardware perspective, we have replaced the faulty mid-plane globally that was causing drive issues for some customers. This also caused some Veeam repositories to suffer drive disconnections, causing jobs to fail.
For the software, we uncovered and reported an issue to Veeam in the way our provider side Veeam B&R servers communicate with the backend repositories over SSH. There is a bug in the way the VBR maintains stale SSH connections to the repositories at scale, keeping the SSH session alive and not tearing it down. The end result of this is that the repositories no longer accept new connections to start the VeeamAgent process to process tenant tasks. We have a workaround for this issue and are contacting customers directly. The workaround we implemented involves controlling the issue reported above to maintain a good state for incoming task processing, allowing customer tenant jobs to process accordingly. We will provide an update once we get a proper patch from Veeam, but so far the workaround is providing good results.
We are also aware that our iland status page is reporting 100% uptime, even with degraded performance. We recognize that limiting our status page to only availability status is not sufficient and does not accurately reflect the service environment. We are committed to improving our service reporting and will update our status page to provide additional detail updates as we resolve these issues.
We will continue to follow up with individual customers directly. Please contact support with any questions or concerns.
UK +(44) (0) 20 7096 0149
SG +(65) 3158 8438
NL +(31) (0) 10 808 0440
AU +(61) (0) 2 9056 7004