Ufdjrw

I am configuring a Hazelcast Multimap without backups (on purpose):

 config.getMultiMapConfig(SESSIONS_MAP)
 .setBackupCount(0)
 .setAsyncBackupCount(0)
 .setValueCollectionType(MultiMapConfig.ValueCollectionType.SET);

My goal is that each instance stores its own values in the MultiMap, so that when a server disappears, those values are lost. Is above configuration correct?

Example: Server instances in a cluster host user sessions. I want to store users in a MultiMap, so that each user is physically stored on the local instance, but other instances can look up where a user session exists. When a server crashes, the user sessions disappear, and so should the entries in the MultiMap. [Users are actually stored in rooms, like MultiMap<roomId, Set<userId>>, where a room may span multiple instances. If one instance goes down, the room may survive, but I want the users on the current instance to become unavailable in the MultiMap as well.]

Only if above is guaranteed: In a controlled shutdown, is it worth to clean up the local entries before shutting down, or is it cheaper to just make the instance disappear?

The manual at https://docs.hazelcast.org/docs/latest-dev/manual/html-single/index.html#configuring-multimap doesn't clearly spell out what actually happens (or I am too blind to find it).

If you set backup counts to zero, it means that each entry will only be stored in one partition (the primary). But it doesn't mean that partition will be hosted on the "local" cluster node.

The partition where any entry is stored is determined by a hashing algorithm, but the mapping of partitions to cluster nodes will change as cluster membership changes (nodes are added or removed). So I don't think trying to manipulate the hashcode is a good way to go.

Since you mention the "local instance", I'm guessing you're using Hazelcast in embedded mode, and the Hazelcast cluster nodes are on the same servers that host the "rooms". You might want to configure a MembershipListener; this listener would be notified whenever a node leaves the cluster, and the listener could then remove map entries related to user sessions hosted in rooms on that node.

Thats a wrong use case for a partition-based distributed system. When you store in a partitioned distributed data-structure such as Map or MultiMap, you do not have control over which partition would host your key-value data. The host partition to your data is determined by consistent hashing algorithm applied on the key. This applies to both - write as well as read operations. And with backup enabled, the data is replicated in backup partitions on each node so that data can be recovered in case of a node failure.

So in your case, you don't even know whether a particular entry is indeed local to your instance (unless you are manually recording this mapping of key-partition using Hazelcast APIs). You are looking up an entry hoping it to be local to that instance because you executed the write operation of that entry from that same node but in reality, that entry may be stored on a partition in some other node in the cluster.

I believe what you want is NearCache which in other words can also be addressed as L1 cache - local to your application. If you loose the app instance, you loose the NearCache and is not available with MultiMap. But even with NearCache, you will never receive "null" or "data not found" because NearCache in principle, loads the data from partition owner (cluster node) if the data is not found in NearCache.

You can also turning off backup but that will mean loosing data on the lost node which may not be local to your application.

Hope that helps.