I am currently facing a challenging problem, and I would appreciate any suggestions on how to optimize it. In the system, we have many emails that need to be polled via POP3 to check for new emails within 10 seconds. Moreover, the contents of these emails need to be updated in the database. These emails may belong to the same person or to different people. Currently, I am using a single-threaded loop, which is quite slow. Is there a way to improve the speed?
One idea I had is to use multi-threading, where each thread is responsible for processing a single email. The issue with this approach is that if the emails belong to the same person, how can we ensure that the updates do not conflict with each other?
Another idea I had is to design a mail processing center with multiple client nodes. These nodes would connect to my center and compete for tasks. Would this design be overkill?