[RESOLVED] NA21 | Service Outage

 POSTED: Nov 11, 2019 11:45:37 AM        NA21, Service Outage

Continue reading...

NA11 | NA21 | EU11 | EU21 Production Unavailable During 19.8 Product Release

 POSTED: Oct 23, 2019 12:17:59 PM        Downtime, NA21, NA11, Product Release, EU11, EU21

SpringCM Production accounts will be unavailable Friday, November 1 from 9pm to Midnight CDT for the 19.8 Product release. We apologize for any inconvenience.

Continue reading...

[RESOLVED] NA21 | Service Degradation

 POSTED: Oct 14, 2019 11:25:43 AM        Production, NA21, Service Outage

[ROOT CAUSE ANALYSIS]

Description:

NA21 Environment Performance Degredation

Timeframe:

2019-10-14 , 08:22AM CT to 08:54AM CT

Impact:

Customers experienced slowness or timeout while navigating SpringCM

Cause:

An unexpected database snapshot was generated for multiple publications which caused a degradation in database performance.

Resolution:

Once the snapshot generation finished performance returned to normal.  We have a support case open with the hosting vendor for investigation of the cause of the snapshot generation.

[RESOLVED] The SpringCM Operations team detected and resolved a service availability issue on the NA21 instance starting at 10:22AM - and ending at 10:54AM CT. During this time, customers may have experienced latency, an inability to access resources, or received an error when loading pages.

Continue reading...

NA11 | NA 21 | EU 11 | EU 21 Production Unavailable During 19.8 Product Release

 POSTED: Oct 7, 2019 3:29:42 PM        Downtime, NA21, NA11, Product Release, EU11, EU21

Continue reading...

NA | EU Production Reporting Maintenance Oct 11 - Oct 14

 POSTED: Oct 1, 2019 12:25:23 PM        NA21, NA11, EU11, EU21

Please be advised that maintenance has been scheduled for the NA11 and NA21 Production Environments for 9pm CDT on Friday October 11 to 12AM CDT on Monday October 14. Additionally, maintenance has been scheduled for the EU11 and EU21 Production Environments for 21:00 UTC on Friday, October 11 to 24:00 UTC on Monday October 13. During the maintenance window customers should not expect reports to fail but will likely return out of date/incomplete data.

Continue reading...

[RESOLVED] NA21 | Service Degradation

 POSTED: Aug 30, 2019 1:39:57 PM        NA21, Service Degradation

[ROOT CAUSE ANALYSIS Sept, 24, 2019] 

Description:

Spring - UI Session Management Intermittent Connection Failures/Outage

Timeframe:

2019-08-30 , 13:00CT to 15:30CT

Impact:

About 10% of (SpringCM) users whose accounts are primary to the NA21 instance of SpringCM got errors trying to login and active sessions got redirected to the login page or received a gateway time out.
This only impacted our (SpringCM's) NA21 user base.

Cause:

During a recent release we changed session management platforms.  There was an increase in requests to the new platform and unanticipated traffic patterns

Resolution:

Modified the maximum number of clients that can be handled simultaneously. Rolled back implementation to the previous version.

[RESOLVED] The SpringCM Technology team identified and resolved the service availability issue on the NA21 instance on Friday, August 30 at approximately 15:30 CST.   The outage lasted approximately from 13:00 CST to 15:30 CST (2.5 hours).  We apologize for the inconvenience and will be providing a Root Cause Analysis in the near future.

Continue reading...

NA11 | NA21 | EU11 | EU21 September 19.7 Product Release

 POSTED: Aug 27, 2019 4:33:33 PM        NA21, NA11, Product Release, EU11, EU21, Zero Downtime

Continue reading...

[UPDATE] NA11 | NA21 | EU11 | EU21 - Production August 19.6 Product Release

 POSTED: Jul 22, 2019 3:30:16 PM        NA21, NA11, Product Release, EU11, EU21

[UPDATE July 25, 2019] - The Production environment will be unavailable during this product release, from Friday, August 9, 9PM - Midnight US Central Time.  We inadvertently reported in the original trust post that we did not anticipate any impact to availability.  We apologize for any inconvenience.

Continue reading...

[RESOLVED] NA11 | NA21 | EU11 | EU21 - Document Generation Performance

 POSTED: Jul 3, 2019 12:33:50 PM        NA21, NA11, Document Creation

Root Cause Analysis:

Description:

Data Inconsistency in NA11, NA21, EU11, EU21

Timeframe:

2019-07-03 12:00PM CDT to 2019-07-04 5:30PM CDT

Impact:

Customers may have experienced inability to generate new documents via the Document Generation feature.

Cause:

A change to distributed database replication settings caused some database replicas not to have the data they needed.

Resolution:

A hotfix was deployed to increase database query consistency levels so clients always get the most current data from the database, and data inconsistencies were repaired.

Next Steps:

We are searching out other queries which may benefit from increasing database query consistency levels, and we're developing a less disruptive procedure for modifying database replication settings.

[UPDATED:  Jul 8, 2019 8:11 AM] -  In addition to the issues experienced on NA11 and NA21 on July 3, the EU11 and EU21 Production environments experienced the same service availability issues on July 4 from approximately 10:00 AM to 6:00 PM CST.  The SpringCM Technology Team isolated and repaired the service availability issue in that time.

[RESOLVED:  Jul 3, 2019 7:06 PM]  The SpringCM Technology Team has isolated and repaired the service availability issue on the NA11 and NA21 instances.  The problem began at approximately 12:20 PM CST and was resolved by 6:06 PMCST.  We apologize for the inconvenience and will be providing a root cause analysis of the issue in a near future.

[UPDATE: Jul 3, 2019 3:52:00 PM] - The SpringCM Technology Team continues to work on isolating the service availability issue on the NA11 and NA21 instances. Please check the status of trust.springcm.com for updates regarding this issue.

Continue reading...

NA11 | NA21 | EU11 | EU21 June 19.5 Product Release

 POSTED: Jun 6, 2019 11:25:07 AM        Production Environment, NA21, NA11, Product Release, EU11, EU21

SpringCM's June 2019 Product Release will be deployed to the Production environments on Friday, June 28th 2019 between 9PM and Midnight, US Central Time.

Continue reading...

Subscribe to Email Updates

Subscribe by email to get notified as soon as we post a status update.