Welcome to TxMongo’s documentation!

What is TxMongo

TxMongo is a pure-Python Twisted MongoDB client library that implements:

  • Asynchronous client driver to MongoDB

Get it from PyPI, find out what’s new in the Changelog!

Quick Usage Example

from OpenSSL import SSL
from txmongo.connection import ConnectionPool
from twisted.internet import defer, reactor, ssl

class ServerTLSContext(ssl.DefaultOpenSSLContextFactory):
    def __init__(self, *args, **kw):
        kw['sslmethod'] = SSL.TLSv1_METHOD
        ssl.DefaultOpenSSLContextFactory.__init__(self, *args, **kw)

@defer.inlineCallbacks
def example():
    tls_ctx = ServerTLSContext(privateKeyFileName='./mongodb.key', certificateFileName='./mongodb.crt')
    mongodb_uri = "mongodb://localhost:27017"

    mongo = yield ConnectionPool(mongodb_uri, ssl_context_factory=tls_ctx)

    foo = mongo.foo  # `foo` database
    test = foo.test  # `test` collection

    # fetch some documents
    docs = yield test.find(limit=10)
    for doc in docs:
        print doc

if __name__ == '__main__':
    example().addCallback(lambda ign: reactor.stop())
    reactor.run()

User’s Guide

txmongo package

Submodules

txmongo.collection module

class txmongo.collection.Collection(database, name, write_concern=None)[source]

Bases: object

aggregate(pipeline, full_response=False)[source]
count(spec=None, fields=None)[source]
create_index(sort_fields, **kwargs)[source]
database
delete_many(filter)[source]
delete_one(filter)[source]
distinct(key, spec=None)[source]
drop(**kwargs)[source]
drop_index(index_identifier)[source]
drop_indexes()[source]
ensure_index(sort_fields, **kwargs)[source]
filemd5(spec)[source]
find(spec=None, skip=0, limit=0, fields=None, filter=None, cursor=False, **kwargs)[source]
find_and_modify(query=None, update=None, upsert=False, **kwargs)[source]
find_one(spec=None, fields=None, **kwargs)[source]
find_one_and_delete(filter, projection=None, sort=None, **kwargs)[source]
find_one_and_replace(filter, replacement, projection=None, sort=None, upsert=False, return_document=False, **kwargs)[source]
find_one_and_update(filter, update, projection=None, sort=None, upsert=False, return_document=False, **kwargs)[source]
find_with_cursor(spec=None, skip=0, limit=0, fields=None, filter=None, **kwargs)[source]

find method that uses the cursor to only return a block of results at a time. Arguments are the same as with find() returns deferred that results in a tuple: (docs, deferred) where docs are the current page of results and deferred results in the next tuple. When the cursor is exhausted, it will return the tuple ([], None)

full_name
group(keys, initial, reduce, condition=None, finalize=None)[source]
index_information()[source]
insert(docs, safe=None, flags=0, **kwargs)[source]
insert_many(documents, ordered=True)[source]
insert_one(document)[source]
map_reduce(map, reduce, full_response=False, **kwargs)[source]
name
options()[source]
remove(spec, safe=None, single=False, flags=0, **kwargs)[source]
rename(new_name)[source]
replace_one(filter, replacement, upsert=False)[source]
save(doc, safe=None, **kwargs)[source]
update(spec, document, upsert=False, multi=False, safe=None, flags=0, **kwargs)[source]
update_many(filter, update, upsert=False)[source]
update_one(filter, update, upsert=False)[source]
with_options(**kwargs)[source]

Get a clone of collection changing the specified settings.

write_concern

txmongo.connection module

class txmongo.connection.ConnectionPool(uri='mongodb://127.0.0.1:27017', pool_size=1, ssl_context_factory=None, **kwargs)[source]

Bases: object

authenticate(database, username, password, mechanism='DEFAULT')[source]
disconnect()[source]
get_default_database()[source]
getprotocol()[source]
getprotocols()[source]
uri
write_concern
class txmongo.connection.MongoConnection(host='127.0.0.1', port=27017, pool_size=1, **kwargs)[source]

Bases: txmongo.connection.ConnectionPool

txmongo.connection.MongoConnectionPool

alias of MongoConnection

txmongo.connection.lazyMongoConnection

alias of MongoConnection

txmongo.connection.lazyMongoConnectionPool

alias of MongoConnection

txmongo.database module

class txmongo.database.Database(factory, database_name, write_concern=None)[source]

Bases: object

authenticate(name, password, mechanism='DEFAULT')[source]

Send an authentication command for this database. mostly stolen from pymongo

collection_names()[source]
command(command, value=1, check=True, allowable_errors=None, **kwargs)[source]
connection
create_collection(name, options=None)[source]
drop_collection(name_or_collection)[source]
name
write_concern

txmongo.filter module

txmongo.filter.ASCENDING(keys)[source]

Ascending sort order

txmongo.filter.DESCENDING(keys)[source]

Descending sort order

txmongo.filter.GEO2D(keys)[source]

Two-dimensional geospatial index http://www.mongodb.org/display/DOCS/Geospatial+Indexing

txmongo.filter.GEO2DSPHERE(keys)[source]

Two-dimensional geospatial index http://www.mongodb.org/display/DOCS/Geospatial+Indexing

txmongo.filter.GEOHAYSTACK(keys)[source]

Bucket-based geospatial index http://www.mongodb.org/display/DOCS/Geospatial+Haystack+Indexing

class txmongo.filter.comment(comment)[source]

Bases: txmongo.filter._QueryFilter

class txmongo.filter.explain[source]

Bases: txmongo.filter._QueryFilter

Returns an explain plan for the query.

class txmongo.filter.hint(index_list_or_name)[source]

Bases: txmongo.filter._QueryFilter

Adds a hint, telling Mongo the proper index to use for the query.

class txmongo.filter.snapshot[source]

Bases: txmongo.filter._QueryFilter

class txmongo.filter.sort(key_list)[source]

Bases: txmongo.filter._QueryFilter

Sorts the results of a query.

txmongo.gridfs module

txmongo.protocol module

Low level connection to Mongo.

This module contains the wire protocol implementation for txmongo. The various constants from the protocol are available as constants.

This implementation requires pymongo so that as much of the implementation can be shared. This includes BSON encoding and decoding as well as Exception types, when applicable.

class txmongo.protocol.Delete[source]

Bases: txmongo.protocol.Delete

class txmongo.protocol.Getmore[source]

Bases: txmongo.protocol.Getmore

class txmongo.protocol.Insert[source]

Bases: txmongo.protocol.Insert

class txmongo.protocol.KillCursors[source]

Bases: txmongo.protocol.KillCursors

exception txmongo.protocol.MongoAuthenticationError[source]

Bases: Exception

class txmongo.protocol.MongoClientProtocol[source]

Bases: twisted.internet.protocol.Protocol

get_request_id()[source]
send(request)[source]
send_DELETE(request)[source]
send_GETMORE(request)[source]
send_INSERT(request)[source]
send_KILL_CURSORS(request)[source]
send_MSG(request)[source]
send_QUERY(request)[source]
send_REPLY(request)[source]
send_UPDATE(request)[source]
class txmongo.protocol.MongoDecoder[source]

Bases: object

dataBuffer = None
static decode(message_data)[source]
feed(data)[source]
next()
class txmongo.protocol.MongoProtocol[source]

Bases: txmongo.protocol.MongoServerProtocol, txmongo.protocol.MongoClientProtocol

authenticate(database_name, username, password, mechanism)[source]
authenticate_mongo_cr(database_name, username, password)[source]
authenticate_scram_sha1(database_name, username, password)[source]
connectionLost(reason=<twisted.python.failure.Failure twisted.internet.error.ConnectionDone: Connection was closed cleanly.>)[source]
connectionMade()[source]
connectionReady()[source]
fail(reason)[source]
get_last_error(db, **options)[source]
handle_REPLY(request)[source]
inflight()[source]
max_wire_version = None
min_wire_version = None
send_GETMORE(request)[source]
send_QUERY(request)[source]
set_wire_versions(min_wire_version, max_wire_version)[source]
class txmongo.protocol.MongoServerProtocol[source]

Bases: twisted.internet.protocol.Protocol

dataReceived(data)[source]
handle(request)[source]
handle_DELETE(request)[source]
handle_GETMORE(request)[source]
handle_INSERT(request)[source]
handle_KILL_CURSORS(request)[source]
handle_MSG(request)[source]
handle_QUERY(request)[source]
handle_REPLY(request)[source]
handle_UPDATE(request)[source]
class txmongo.protocol.Msg(len, request_id, response_to, opcode, message)

Bases: tuple

len

Alias for field number 0

message

Alias for field number 4

opcode

Alias for field number 3

request_id

Alias for field number 1

response_to

Alias for field number 2

class txmongo.protocol.Query[source]

Bases: txmongo.protocol.Query

class txmongo.protocol.Reply[source]

Bases: txmongo.protocol.Reply

class txmongo.protocol.Update[source]

Bases: txmongo.protocol.Update

Module contents

txmongo._gridfs package

Submodules

txmongo._gridfs.errors module

Exceptions raised by the gridfs package

exception txmongo._gridfs.errors.CorruptGridFile[source]

Bases: txmongo._gridfs.errors.GridFSError

Raised when a file in GridFS is malformed.

exception txmongo._gridfs.errors.GridFSError[source]

Bases: Exception

Base class for all GridFS exceptions.

New in version 1.5.

exception txmongo._gridfs.errors.NoFile[source]

Bases: txmongo._gridfs.errors.GridFSError

Raised when trying to read from a non-existent file.

New in version 1.6.

exception txmongo._gridfs.errors.UnsupportedAPI[source]

Bases: txmongo._gridfs.errors.GridFSError

Raised when trying to use the old GridFS API.

In version 1.6 of the PyMongo distribution there were backwards incompatible changes to the GridFS API. Upgrading shouldn’t be difficult, but the old API is no longer supported (with no deprecation period). This exception will be raised when attempting to use unsupported constructs from the old API.

New in version 1.6.

txmongo._gridfs.grid_file module

Tools for representing files stored in GridFS.

class txmongo._gridfs.grid_file.GridIn(root_collection, **kwargs)[source]

Bases: object

Class to write data to GridFS.

chunk_size

Chunk size for this file.

This attribute is read-only.

close()[source]

Flush the file and close it.

A closed file cannot be written any more. Calling close() more than once is allowed.

closed

Is this file closed?

content_type

Mime-type for this file.

This attribute can only be set before close() has been called.

filename

Name of this file.

This attribute can only be set before close() has been called.

length

Length (in bytes) of this file.

This attribute is read-only and can only be read after close() has been called.

md5

MD5 of the contents of this file (generated on the server).

This attribute is read-only and can only be read after close() has been called.

upload_date

Date that this file was uploaded.

This attribute is read-only and can only be read after close() has been called.

write(data)[source]

Write data to the file. There is no return value.

data can be either a string of bytes or a file-like object (implementing read()).

Due to buffering, the data may not actually be written to the database until the close() method is called. Raises ValueError if this file is already closed. Raises TypeError if data is not an instance of str or a file-like object.

Parameters:
  • data: string of bytes or file-like object to be written to the file
writelines(sequence)[source]

Write a sequence of strings to the file.

Does not add separators.

class txmongo._gridfs.grid_file.GridOut(root_collection, doc)[source]

Bases: object

Class to read data out of GridFS.

aliases

List of aliases for this file.

This attribute is read-only.

chunk_size

Chunk size for this file.

This attribute is read-only.

close()[source]
content_type

Mime-type for this file.

This attribute is read-only.

length

Length (in bytes) of this file.

This attribute is read-only.

md5

MD5 of the contents of this file (generated on the server).

This attribute is read-only.

metadata

Metadata attached to this file.

This attribute is read-only.

name

Name of this file.

This attribute is read-only.

read(size=-1)[source]

Read at most size bytes from the file (less if there isn’t enough data).

The bytes are returned as an instance of str. If size is negative or omitted all data is read.

Parameters:
  • size (optional): the number of bytes to read
seek(pos, whence=0)[source]

Set the current position of this file.

Parameters:
  • pos: the position (or offset if using relative positioning) to seek to
  • whence (optional): where to seek from. os.SEEK_SET (0) for absolute file positioning, os.SEEK_CUR (1) to seek relative to the current position, os.SEEK_END (2) to seek relative to the file’s end.
tell()[source]

Return the current position of this file.

upload_date

Date that this file was first uploaded.

This attribute is read-only.

class txmongo._gridfs.grid_file.GridOutIterator(grid_out, chunks)[source]

Bases: object

next()

Module contents

GridFS is a specification for storing large objects in Mongo.

The gridfs package is an implementation of GridFS on top of pymongo, exposing a file-like interface.

class txmongo._gridfs.GridFS(database, collection='fs')[source]

Bases: object

An instance of GridFS on top of a single Database.

delete(file_id)[source]

Delete a file from GridFS by "_id".

Removes all data belonging to the file with "_id": file_id.

Warning

Any processes/threads reading from the file while this method is executing will likely see an invalid/corrupt file. Care should be taken to avoid concurrent reads to a file while it is being deleted.

Parameters:
  • file_id: "_id" of the file to delete

New in version 1.6.

get(file_id)[source]

Get a file from GridFS by "_id".

Returns an instance of GridOut, which provides a file-like interface for reading.

Parameters:
  • file_id: "_id" of the file to get

New in version 1.6.

get_last_version(filename)[source]

Get a file from GridFS by "filename".

Returns the most recently uploaded file in GridFS with the name filename as an instance of GridOut. Raises NoFile if no such file exists.

An index on {filename: 1, uploadDate: -1} will automatically be created when this method is called the first time.

Parameters:
  • filename: "filename" of the file to get

New in version 1.6.

list()[source]

List the names of all files stored in this instance of GridFS.

Changed in version 1.6: Removed the collection argument.

new_file(**kwargs)[source]

Create a new file in GridFS.

Returns a new GridIn instance to which data can be written. Any keyword arguments will be passed through to GridIn().

Parameters:
  • **kwargs (optional): keyword arguments for file creation

New in version 1.6.

put(data, **kwargs)[source]

Put data in GridFS as a new file.

Equivalent to doing:

>>> f = new_file(**kwargs)
>>> try:
>>>     f.write(data)
>>> finally:
>>>     f.close()

data can be either an instance of str or a file-like object providing a read() method. Any keyword arguments will be passed through to the created file - see GridIn() for possible arguments. Returns the "_id" of the created file.

Parameters:
  • data: data to be written as a file.
  • **kwargs (optional): keyword arguments for file creation

New in version 1.6.

Meta

Changelog

Release 15.2.1 (2015-09-07)

Bugfix release to handle uncaught exceptions in logging and to remove support for python 2.6 and since it was removed in latest Twisted.

Release 15.2 (2015-09-05)

This release makes TxMongo fully Python3 compatible and has an API change that breaks older TxMongo compatibility by bringing it inline with PyMongo.

API Changes
  • txmongo.dbref removed. Use bson.dbref instead. Incompatibility note: bson.dbref.DBRef takes collection name as string while txmongo.dbref.DBRef was able to accept Collection instance. Please use collection.name instead.
  • Added timeout parameter for connection.ConnectionPool that can passed on to Twisted’s connectTCP and connectSSL methods.
Features
  • name, full_name and database properties of Collection
  • Python3 compatible.

Release 15.1 (2015-06-08)

This is a major release in that while increasing code coverage to 95% ( see https://coveralls.io/builds/2749499 ), we’ve also caught several bugs, added features and changed functionality to be more inline with PyMongo.

This is no small thanks to travis-ci and coveralls while using tox to cover all iterations that we support.

We can officially say that we are Python 2.6, 2.7 and PyPy compatible.

API Changes
  • TxMongo now requires PyMongo 3.x, if you need PyMongo 2.x support, please use 15.0, otherwise it is highgly recommend to use PyMongo 3.x which still support MongoDB 2.6.
  • Better handling of replica-sets, we now raise an autoreconnect when master is unreachable.
  • Changed the behaviour of find_one to return None instead of an empty dict {} when no result is found.
  • New-style query methods: insert_one/many, update_one/many, delete_one/many, replace_one and find_one_and_update/replace
Features
  • Added db.command function, just like PyMongo.
  • Added support for named indexes in filter.
  • insert(), update(), save() and remove() now support write-concern options via named args: w, wtimeout, j, fsync. safe argument is still supported for backward compatibility.
  • Default write-concern can be specified for Connection using named arguments in constructor or by URI options.
  • Write-concern options can also be set for Database and Collection with write_concern named argument of their constructors. In this case write-concern is specified by instance of pymongo.write_concern.WriteConcern
  • txmongo.protocol.INSERT_CONTINUE_ON_ERROR flag defined for using with insert()
  • Replaced all traditional deferred callbacks (and errbacks) to use @defer.inlineCallbacks
Bugfixes
  • Fixed typo in map_reduce() when returning results.
  • Fixed hang in create_collection() in case of error.
  • Fixed typo in rename() that wasn’t using the right factory.
  • Fixed exception in drop_index that was being thrown when dropping a non-existent collection. This makes the function idempotent.
  • Fixed URI prefixing when “mongodb://” is not present in URI string in connection.
  • Fixed fail-over when using replica-sets in connection. It now raises autoreconnect when there is a problem with the existing master. It is then up to the client code to reconnect to the new master.
  • Fixed number of cursors in protocol so that it works with py2.6, py2.6 and pypy.

Release 15.0 (2015-05-04)

This is the first release using the Twisted versioning method.

API Changes
  • collections.index_information now mirrors PyMongo’s method.
  • getrequestid is now get_request_id
Features
Bugfixes
  • Fixed failing tests due to changes in Python in 2.6
  • Fixed limit not being respected, which should help performance.
  • Find now closes MongoDB cursors.
  • Fixed ‘hint’ filter to correctly serialize with double dollar signs.
Improved Documentation

Release 0.6 (2015-01-23)

This is the last release in this version scheme, we’ll be switching to the Twisted version scheme in the next release.

API Changes
  • TxMongo: None
Features
  • Added SSL support using Twisted SSLContext factory
  • Added “find with cursor” like pymongo
  • Test coverage is now measured. We’re currently at around 78%.
Bugfixes
  • Fixed import in database.py

Release 0.5 (2014-10-02)

Code review and cleanup

Bugfixes
  • Bug fixes

Release 0.4 (2013-01-07)

Significant performance improvements.

API Changes
  • TxMongo: None
Features
  • Support AutoReconnect to connect to fail-over master.
  • Use pymongo instead of in-tree copy.
Bugfixes
  • Bug fixes

Release 0.3 (2010-09-13)

Initial release.

License
  • Apache 2.0

Status and History

TxMongo was created by Alexandre Fiori who developed it during the years 2009-2010. From 2010 and onwards mainly bug fixes were added, as Alexandre entered maintance mode with many contributions being made by others. Development picked back up in 2014 by Bret Curtis with Alexandre’s consent and was migrated to Twisted where it is a first-party Twisted library. TxMongo can be found here:

https://github.com/twisted/txmongo

The MongoDB client library functionality is in active use. It is stable and works very well.

Contributions

How to Contribute

Head over to: https://github.com/twisted/TxMongo and submit your bugs or feature requests. If you wish to contribute code, just fork it, make a branch and send us a pull request. We’ll review it, and push back if necessary.

TxMongo generally follows the coding and documentation standards of the Twisted project.

Contributors

  • 10gen, Inc
  • Alexandre Fiori
  • Alexey Palazhchenko (AlekSi)
  • Amplidata
  • Andre Ferraz
  • Bret Curtis
  • Carl D’Halluin (Amplidata)
  • Christian Hergert
  • Dave Peticolas
  • Gleicon Moraes
  • Ilya Skriblovsky
  • Jonathan Stoppani
  • Mark L
  • Mike Dirolf (mdirolf)
  • Renzo Sanchez-Silva (rnz0)
  • Runar Petursson
  • Silas Sewell
  • Stiletto
  • Toby Padilla
  • Tryggvi Björgvinsson
  • Vanderson Mota (chunda)
  • flanked
  • renzo
  • shylent

Indices and tables