bulljs 를 사용할때 redis memory를 관리하기

bulljs는 nodejs에서 사용하는 redis 기반의 큐 시스템입니다

 

redis를 사용하기로 결정한 이상 redis의 메모리도 잘 관리를 해야합니다

 

redis에서 중요한 설정중 하나인

 

maxmemory-policy

 

• noeviction: New values aren’t saved when memory limit is reached. When a database uses replication, this applies to the primary database
• allkeys-lru: Keeps most recently used keys; removes least recently used (LRU) keys
• allkeys-lfu: Keeps frequently used keys; removes least frequently used (LFU) keys
• volatile-lru: Removes least recently used keys with the expire field set to true.
• volatile-lfu: Removes least frequently used keys with the expire field set to true.
• allkeys-random: Randomly removes keys to make space for the new data added.
• volatile-random: Randomly removes keys with expire field set to true.
• volatile-ttl: Removes keys with expire field set to true and the shortest remaining time-to-live (TTL) value.

위와 같은 설정값들이 있으니 확인을 하시고

 

메모리를 잘 관리하여야 합니다

 

 

그리고 bull을 사용할때있어서도 메모리 유지를 위해 몇가지 활용할수 있는 방법이 있습니다

 

삭제 없이 계속 bull을 이용해서 큐를 사용하면 계속 메모리가 차기만 하고 비워지지 않을테니말이죠

 

job 옵션은 Queue를 생성할때 전역으로 미리 설정할수도 있고

 

개별 job에 대해서도 설정을 할수 있습니다

 

다음과 같은값들을 가지고 있는데요

interface JobOpts {
  priority: number; // Optional priority value. ranges from 1 (highest priority) to MAX_INT  (lowest priority). Note that
  // using priorities has a slight impact on performance, so do not use it if not required.

  delay: number; // An amount of milliseconds to wait until this job can be processed. Note that for accurate delays, both
  // server and clients should have their clocks synchronized. [optional].

  attempts: number; // The total number of attempts to try the job until it completes.

  repeat: RepeatOpts; // Repeat job according to a cron specification, see below for details.

  backoff: number | BackoffOpts; // Backoff setting for automatic retries if the job fails, default strategy: `fixed`.
  // Needs `attempts` to be set.

  lifo: boolean; // if true, adds the job to the right of the queue instead of the left (default false)
  timeout: number; // The number of milliseconds after which the job should fail with a timeout error [optional]

  jobId: number | string; // Override the job ID - by default, the job ID is a unique
  // integer, but you can use this setting to override it.
  // If you use this option, it is up to you to ensure the
  // jobId is unique. If you attempt to add a job with an id that
  // already exists, it will not be added (see caveat below about repeatable jobs).

  removeOnComplete: boolean | number | KeepJobs; // If true, removes the job when it successfully
  // completes. A number specified the amount of jobs to keep. Default behavior is to keep the job in the completed set.
  // See KeepJobs if using that interface instead.

  removeOnFail: boolean | number | KeepJobs; // If true, removes the job when it fails after all attempts. A number specified the amount of jobs to keep, see KeepJobs if using that interface instead.
  // Default behavior is to keep the job in the failed set.
  stackTraceLimit: number; // Limits the amount of stack trace lines that will be recorded in the stacktrace.
}

 

메모리의 확보와 관련있는 파라메터는 이것입니다

interface JobOpts {
  removeOnComplete: boolean | number | KeepJobs; 
  removeOnFail: boolean | number | KeepJobs;
}

removeOnComplete는 말그대로 성공시에 삭제

 

removeOnFail은 실패시에 삭제 입니다

 

boolean 값을 받지만 number값도 받는데요

 

number값을 넣으면 해당 갯수까지만 job의 갯수를 유지합니다

 

이 값을 100으로 설정한경우 101번째 job 데이터가 쌓이면 맨앞의 1번 job 데이터가 삭제되는 셈입니다

 

 

 

그리고 clean 이라는 함수도 존재하는데요

 

clean(grace: number, status?: string, limit?: number): Promise<number[]>

 

 

queue.on('cleaned', function (jobs, type) {
  console.log('Cleaned %s %s jobs', jobs.length, type);
});

//cleans all jobs that completed over 5 seconds ago.
await queue.clean(5000);
//clean all jobs that failed over 10 seconds ago.
await queue.clean(10000, 'failed');

사용법에도 설명이 되어있듯이

 

5000(ms)를 인자로 전달하면 5초가 지난 job data들을 삭제한다는 의미입니다

 

수동으로 job 데이터를 삭제하여 메모리를 확보할수 있는 방법입니다