Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] After manually deleted redis keys, some of the features like metrics, inspector aren't working properly #901

Open
sailesh2341 opened this issue Jul 10, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@sailesh2341
Copy link

Describe the bug
I have manually deleted one of the key from redis DB to remove one of the task under archive state from queue. After doing this asynqmon metrics stopped working and Inspector methods aren't working to get info of tasks/queue.
The error I'm getting both for metrics and Inspector methods are as follows:

asynq UNKNOWN: UNKNOWN: redis eval error: ERR Error running script (call to f_0c80fc1868e3aae205155ae2017ce995a3a7cec0): @user_script:30: user_script:30: attempt to perform arithmetic on local 'bytes' (a boolean value)

To Reproduce
I've tried many ways to resolve this error:

  1. Restarted the application to restart the asynq server and other instances.
  2. Tried to manually insert the deleted key into redis DB again.
  3. Tried moving some task to archive state and back to pending state again so that the lost keys can be recreated.

Nothing of these approaches worked. Since this issue was on production environment , i cannot flush all the redis DB data nor i cannot take any risky actions .
Has anyone else faced the same issue?
Can someone please tell me how do we resolve this issue?

@sailesh2341 sailesh2341 added the bug Something isn't working label Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants