Interesting read, but it seems like the grouping challenge could be fairly trivially solved in the application layer by registering job types (groups) with the poller, and having the round robin logic there.
Right I’m saying you implement the specific round robin logic that you want in the app layer.
Obviously if they don’t want to work 10 Alice jobs in a row you wouldn’t write it that way.
If you want true round robin it's as simple as an ordered list, where you look at the last completed job type, and find the next job type in the list, grab it and go. It's also more explicit this way vs that fairly complex query that is just doing implicit round robin.
So you get the list of potentially hundreds of people from the database, and then loop and do thousands of queries to the database? Query for Alice get 1, query for Bob get 1, query for Charlie there's none, query for Damien there's none, query for Emily get 1, ... eventually you have 100 and you send that many queries again to mark them as RUNNING?
tbh it really irks me that you can read this well-written, 3000-word article and go "I would just solve it trivially by not doing that on the server or something". There's a reason this has to run in the database.
No I would not do something so obviously silly. I’m not sure why you’re making up dumb scenarios as a way to discredit my idea.
I’m not trying to discredit the article or trivialize it. I’m trying to have a discussion about a thought I had while reading, based on my own lived experience working with DB backed queues.
You say this but you did not give many clues as to what your thought actually is. I keep prodding but I'm getting nothing back except for those "no obviously I wouldn't do that".
Will you actually describe your thought or does it not fit in the margin of your comment?
You have a list of queue groups. You write code to round robin through them. Each time, you see if there is a job to be processed for that group. If there is, process it. If there's not, move to the next group in the list.
