The problem that disrupted campus email and web services during separate outages last week, first on the afternoon of Jan. 10 and again on the morning of Jan. 11, has been identified and solved.

The basic cause was a bug in storage area network software code at a Sacramento data center. The bug allowed background data-processing jobs to take precedence over active requests for service. This meant, for example, that a web request to view the main campus website could not access the data needed to fulfill the request.

Similar deferrals involving other campus services eventually caused the outage to spread. The disruption extended to authentication services, which meant people could not sign in to access their campus email accounts.

Normally, redundant systems and processes would have prevented the problem from disrupting services, but the bug also kept failover procedures from engaging.

Information and Educational Technology, working with the software vendor, researched and stabilized the problem Friday afternoon. The problem returned in a milder form Saturday morning. IET made further adjustments, and services have functioned normally since Saturday afternoon. A software upgrade this week will remove the bug.

If you notice problems, please contact the IT Express Computing Services Help Desk at 530-754-HELP (4357). The current status of campus services is posted at status.ucdavis.edu, and IET also posts significant service updates on a Twitter account, @UCDavisStatus.