C# how to deal with sessions left open

C# how to deal with sessions left open

I have a ConcurrentDictionary on server side holding on to all the
<SessionId, UserSessions> pairs.
When a new connection is established a cookie is assigned to a client
browser, perm or temp depending on the RememberMe option.
When clients call the LogOut function it removes the session from the
dictionary.
However, when the client browser is simply closed or crashed, and cookie
was per session or expired or deleted, the server side session object in
memory is remained in dictionary and becomes a ghost. Over time these
ghosts will stack up.
My question is, how to improve the design so that the dead sessions can be
cleaned up after they are expired?
I thought about making a timer service running a cleaning schedule, but it
feels not elegent. Is there a simpler way to do this without depending on
an external service?
My setup is .NET 4.5 and IIS8