What happens to session information about an xDB Contact if MongoDB is unavailable when the session end event fires and Sitecore attempts to flush it to the collections database?
That is a great question, and one a client asked me earlier this week. Since we always recommend highly-available solutions to our clients, including MongoDB replica sets, it’s an edge case we haven’t encountered and honestly, one I hadn’t considered.
After about five minutes of unfruitful googling, I figured I’d reach out to the Queen of Sitecore Documentation herself, Martina Welander (who recently wrote a FANTASTIC article on session state, by the way!).
Naturally, she delivered! By reaching out to the product team, she received this response from Akinori Taira:
In the event that the collections database is unavailable, there is a special ‘Submit Queue’ mechanism that flushes captured data to the local hard drive (the ‘Data\Submit Queue’ folder by default). When the collections database comes back online, a background worker process submits the data from the ‘Submit Queue’ on disk.
As with most everything else in Sitecore, this mechanism is controlled by entries, scattered across a few config files.
In Sitecore 8.0 (and its updates) there is a setting called Analytics.FailOnDatabaseErrors in Sitecore.Analytics.config:
<!-- ANALYTICS FAIL ON DATABASE ERRORS Note: This is a diagnostics setting that should not be enabled on a live system. If set to true, web pages crash and show exception details in case an operation cannot access the database. If set to false, exceptions are suppressed and logged, and visit data is queued until the database is back up. See <submitQueue> section. Default: false --> <setting name="Analytics.FailOnDatabaseErrors" value="false" />
However, in Sitecore 8.1+, the property Analytics.UseSubmitQueue is used from Sitecore.Analytics.Tracking.config:
<!-- ANALYTICS USE SUBMIT QUEUE Specifies if the submit queue should be used when the contact or session submit operations cannot access the database. If set to true, the contact or session is queued until the database is backed up (see the <submitQueue> section). Default: true --> <setting name="Analytics.UseSubmitQueue" value="true" />
Finally, from the Sitecore.Analytics.Tracking.config and Sitecore.Analytics.Tracking.Database.config, you get the following:
<submitQueue> <queue type="Sitecore.Analytics.Data.DataAccess.SubmitQueue.FileSubmitQueue, Sitecore.Analytics" singleInstance="true" /> <backgroundService type="Sitecore.Analytics.SubmitQueueService, Sitecore.Analytics"> <!-- Service wakeup interval in seconds. --> <Interval>60</Interval> </backgroundService> </submitQueue>
I have not tested this failover scenario myself yet, so your mileage may vary, but feel free to comment if you have and how well it works.
Happy Sitecore trails, my friends!
CitizenSitecore 
We had an issue recently where we ran windows update on our MongoDB and rebooted it, which I thought would be ok because I verified all of the submitQueue settings were true and setup as indicated above…so I assumed it would fallback to using the SubmitQueue. That did not seem to work and this is the error that everyone saw who visited out website:
[SocketException (0x274d): No connection could be made because the target machine actively refused it 10.0.0.22:27017]
System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress) +520
System.Net.Sockets.Socket.Connect(EndPoint remoteEP) +264
System.Net.Sockets.TcpClient.Connect(IPEndPoint remoteEP) +84
MongoDB.Driver.Internal.MongoConnection.Open() +211
MongoDB.Driver.Internal.MongoConnection.SendMessage(BsonBuffer buffer, Int32 requestId) +321
MongoDB.Driver.Internal.MongoConnection.SendMessage(MongoRequestMessage message) +134
MongoDB.Driver.Operations.CommandOperation`1.Execute(MongoConnection connection) +244
MongoDB.Driver.MongoServerInstance.Ping(MongoConnection connection) +347
MongoDB.Driver.MongoServerInstance.Connect() +507
MongoDB.Driver.Internal.DirectMongoServerProxy.Connect(TimeSpan timeout, ReadPreference readPreference) +348
[MongoConnectionException: Unable to connect to server session.mongoserver.us:27017: No connection could be made because the target machine actively refused it 10.0.0.22:27017.]
MongoDB.Driver.Internal.DirectMongoServerProxy.Connect(TimeSpan timeout, ReadPreference readPreference) +1051
MongoDB.Driver.Internal.DirectMongoServerProxy.ChooseServerInstance(ReadPreference readPreference) +112
MongoDB.Driver.MongoServer.AcquireConnection(ReadPreference readPreference) +249
MongoDB.Driver.MongoCollection.Update(IMongoQuery query, IMongoUpdate update, MongoUpdateOptions options) +129
Sitecore.SessionProvider.MongoDB.MongoSessionStateStore.InsertItem(String application, String id, Int32 flags, SessionStateStoreData sessionState) +745
System.Web.SessionState.SessionStateModule.OnReleaseState(Object source, EventArgs eventArgs) +892
System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +229
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +98
LikeLike
Chris, that’s odd – that shouldn’t happen. I’d suggest opening a support ticket with Sitecore.
LikeLike
It seems like the fragile setting is the custom sessionState in the web.config for storing the private session state data. When that becomes unavailable, IIS freaks out and throws that error.
We set our load balancers back to sticky sessions then I updated IIS to use an InProc sessionState for the private sessions.
When I left the other settings to use MongoDB (for the Shared Sessions and hosting the xDB collections), I could successfully shutdown the MongoD service and I saw files queue up in the SubmitQueue folder.
LikeLike
Oh, I see – sorry, I misread your previous comment So, the issue isn’t with WFFM queuing form data to the SubmitQueue while MongoDB is unavailable, it’s because you have session configured to use the mongoSessionStateProvider. Do you have just a single MongoDB server or have you configured a replica set for high availability?
LikeLike
I wasn’t testing anything with WFFM, only xDB tracking data. We did not have a replica set, so the entire database was unavailable when we rebooted. We will be creating a replica set now.
After reading about the SubmitQueue, I had thought any xDB analytics tracking would queue up changes in the server’s SubmitQueue if it couldn’t connect to the MongoDB database. I’ve only validated this on a Test server that doesn’t have a lot of activity, so I didn’t see a lot of files queue up in there, but it seemed to work that way for xDB data when I made the MongoDB database temporarily unavailable.
LikeLike