Welcome to job_progress’s documentation!¶
Contents:
Quickstart¶
Installation¶
Setup¶
Here’s how to setup job_progress:
# mypackage/jobprogress.py
from __future__ import absolute_import
from job_progress import JobProgress, Session
from job_progress.backends.redis import RedisBackend
TEST_CONFIG = {
'backend_url': 'redis://localhost:6379/0',
'heartbeat_enabled': True,
}
session = Session(backend=RedisBackend(TEST_CONFIG))
JobProgress.set_session(session)
Storing progress¶
You can then use it from other files:
from job_progress import states
from mypackage.job_progress import JobProgress
def toast_bread(toasts):
"""Toast some bread."""
# You can store any meta data on the job, and provide a job identifier.
job = JobProgress(id_='toaster_job_1'
data={'created_at': '2014-12-12'},
amount=len(toasts), # Amount of work
state=states.STARTED)
for toast in toasts:
try:
# Toast bread, then
job.add_one_success()
except:
job.add_failure()
Getting progress¶
If you have a specific job id, you can get the job directly:
from mypackage.job_progress import session
job = session.get('toaster_job_1')
job.get_progress()
# Return something like:
# {'FAILURE': 1, 'PENDING': 8, 'SUCCESS': 1}
Jobs are also indexed:
from job_progress import states
from mypackage.job_progress import session
jobs = session.query(state=states.STARTED)
Setting state¶
You can set the state of a specific job:
from job_progress import states, session
job = session.get('31bd07a2-5174-4cbd-a575-d68382518f20')
job.state = states.SUCCESS
Example session¶
>>> from job_progress.job_progress import JobProgress
>>> from job_progress.backends.redis import RedisBackend
>>> from job_progress import states, session
>>> settings = {"url": "redis://localhost:6379/0"}
>>> JobProgress.backend = RedisBackend(settings)
>>> JobProgress.session = session.Session()
>>> # 10 is the amount of work
>>> job = JobProgress({"created_at": "2013-12-12"}, amount=10, state=states.STARTED)
>>> job.add_one_success()
>>> job.add_one_failure()
>>> job.get_progress()
{'FAILURE': 1, 'PENDING': 8, 'SUCCESS': 1}
>>> del job
>>> # Jobs are indexed
>>> jobs = JobProgress.query(state=states.STARTED)
>>> jobs[0].get_progress()
{'FAILURE': 1, 'PENDING': 8, 'SUCCESS': 1}
Example Integration With Celery¶
Foreword¶
The main idea behind this library is to be able to store a job’s metadata and progress into a backend, so as to be able to retrieve them.