𝗟𝗼𝗮𝗱 𝗕𝗮𝗹𝗮𝗻𝗰𝗶𝗻𝗴 – spreads incoming traffic across nodes to avoid bottlenecks.
𝗟𝗼𝗮𝗱 𝗟𝗲𝘃𝗲𝗹𝗶𝗻𝗴 – smooths out spikes by queuing work for later processing.
𝗟𝗼𝗮𝗱 𝗦𝗵𝗲𝗱𝗱𝗶𝗻𝗴 – drops non-critical requests to keep the core alive.
Pick the wrong one, and you’re either wasting resources or crashing hard.
𝗪𝗵𝗲𝗻 𝗧𝗼 𝗨𝘀𝗲 𝗟𝗢𝗔𝗗 𝗕𝗔𝗟𝗔𝗡𝗖𝗜𝗡𝗚
→ Evenly distribute requests to ensure consistent response times.
→ Scale horizontally during peak hours (e.g. market open/close).
→ Prevent single points of failure with smart routing.
Distributes traffic across servers to prevent overload and keep performance steady. Smart routing avoids bottlenecks and single points of failure. Works best when resources are healthy and scalable.
𝗪𝗵𝗲𝗻 𝗧𝗼 𝗨𝘀𝗲 𝗟𝗢𝗔𝗗 𝗟𝗘𝗩𝗘𝗟𝗜𝗡𝗚
→ Request peaks that come in waves? Buffer them.
→ Use queues to decouple services (e.g., order matching from settlement).
→ Works when a little delay is acceptable for long-term system health.
A messaging channel is set up between clients and the service. This channel helps manage the flow for requests, allowing the service to handle them at its own pace. But beware of queue growth and latency creep.
𝗪𝗵𝗲𝗻 𝗧𝗼 𝗨𝘀𝗲 𝗟𝗢𝗔𝗗 𝗦𝗛𝗘𝗗𝗗𝗜𝗡𝗚
→ Market crash? Order spike? Can't handle everything?
→ Prioritize order execution over non-critical analytics or notifications.
→ Drop 20% of traffic if it means saving the core 80%.
A server can become overwhelmed with requests, leading to slow performance or ever unavailability. To manage this, it can reject excess requests when it reaches its capacity. Better to shed load than go down completely.
𝗪𝗵𝗲𝗿𝗲 𝗜𝘁 𝗔𝗹𝗹 𝗖𝗼𝗺𝗲𝘀 𝗧𝗼𝗴𝗲𝘁𝗵𝗲𝗿
In a stock exchange system:
• Balance incoming traffic to keep nodes healthy.
• Shed non-critical updates during volatile spikes.
• Level settlement processing using queues to avoid crash.
𝗪𝗵𝗲𝗿𝗲 𝗧𝗲𝗮𝗺𝘀 𝗢𝗳𝘁𝗲𝗻 𝗚𝗲𝘁 𝗜𝘁 𝗪𝗿𝗼𝗻𝗴
• Relying only on balancing—when all nodes are overloaded, it fails.
• Over-shedding—leads to lost revenue and frustrated users.
• Leveling without limits—queues grow endlessly, latency explodes.
Load isn’t the problem. Mismanaging it is.