PK ‡|Ô@\ª0oj† j† eulcommon-0.17.0/binfile.html
Map binary data on-disk to read-only Python objects.
This module facilitates exposing stored binary data using common Pythonic idioms. Fields in relocatable binary objects map to Python attributes using a priori knowledge about how the binary structure is organized. This is akin to the standard struct module, but with some slightly different use cases. struct, for instance, offers a more terse syntax, which is handy for certain simple structures. struct is also a bit faster since it’s implemented in C. This module’s more verbose BinaryStructure definitions give it a few advantages over struct, though:
- This module allows users to define their own field types, where struct field types are basically inextensible.
- The object-based nature of BinaryStructure makes it easy to add non-structural properties and methods to subclasses, which would require a bit of reimplementing and wrapping from a struct tuple.
- BinaryStructure instances access fields through named properties instead of indexed tuples. struct tuples are fine for structures a few fields long, but when a packed binary structure grows to dozens of fields, navigating its struct tuple grows perilous.
- BinaryStructure unpacks fields only when they’re accessed, allowing us to define libraries of structures scores of fields long, understanding that any particular application might access only one or two of them.
- Fields in a BinaryStructure can overlap eachother, greatly simplifying both C unions and fields with multiple interpretations (integer/string, signed/unsigned).
- This module makes sparse structures easy. If you’re reverse-engineering a large binary structure and discover a 4-byte integer in the middle of 68 bytes of unidentified mess, this module makes it easy to add an IntegerField at a known structure offset. struct requires you to split your '68x' into a '32xI32x' (or was that a '30xi34x'? Better recount.)
Suppose we have an 8-byte file whose binary data consists of the bytes 0, 1, 2, 3, etc.:
>>> with open('numbers.bin') as f:
... f.read()
...
'\x00\x01\x02\x03\x04\x05\x06\x07'
Suppose further that these contents represent sensible binary data, laid out such that the first two bytes are a literal string value. Except that sometimes, in the binary format we’re parsing, it might sometimes be necessary to interpret those first two bytes not as a literal string, but instead as a number, encoded as a big-endian unsigned integer. Following that is a variable-length string, encoded with the total string length in the third byte.
This structure might be represented as:
from eulcommon.binfile import *
class MyObject(BinaryStructure):
mybytes = ByteField(0, 2)
myint = IntegerField(0, 2)
mystring = LengthPrepededStringField(2)
Client code might then read data from that file:
>>> f = open('numbers.bin')
>>> obj = MyObject(f)
>>> obj.mybytes
'\x00\x01'
>>> obj.myint
1
>>> obj.mystring
'\x03\x04'
It’s not uncommon for such binary structures to be repeated at different points within a file. Consider if we overlay the same structure on the same file, but starting at byte 1 instead of byte 0:
>>> f = open('numbers.bin')
>>> obj = MyObject(f, offset=1)
>>> obj.mybytes
'\x01\x02'
>>> obj.myint
258
>>> obj.mystring
'\x04\x05\x06'
A superclass for binary data structures superimposed over files.
Typical users will create a subclass containing field objects (e.g., ByteField, IntegerField). Each subclass instance is created with a file and with an optional offset into that file. When code accesses fields on the instance, they are calculated from the underlying binary file data.
Instead of a file, it is occasionally appropriate to overlay an mmap structure (from the mmap standard library). This happens most often when one BinaryStructure instance creates another, passing self.mmap to the secondary object’s constructor. In this case, the caller may specify the mm argument instead of an fobj.
Parameters: |
|
---|
A field mapping fixed-length binary data to Python strings.
Parameters: |
|
---|
Typical users will create a ByteField inside a BinaryStructure subclass definition:
class MyObject(BinaryStructure):
myfield = ByteField(0, 4) # the first 4 bytes of the file
When you instantiate the subclass and access the field, its value will be the literal bytes at that location in the structure:
>>> o = MyObject('file.bin')
>>> o.myfield
'ABCD'
A field mapping variable-length binary strings to Python strings.
This field accesses strings encoded with their length in their first byte and string data following that byte.
Parameters: | offset – The offset of the single-byte string length. |
---|
Typical users will create a LengthPrependedStringField inside a BinaryStructure subclass definition:
class MyObject(BinaryStructure):
myfield = LengthPrependedStringField(0)
When you instantiate the subclass and access the field, its length will be read from that location in the structure, and its data will be the bytes immediately following it. So with a file whose first bytes are '\x04ABCD':
>>> o = MyObject('file.bin')
>>> o.myfield
'ABCD'
A field mapping fixed-length binary data to Python numbers.
This field accessses arbitrary-length integers encoded as binary data. Currently only big-endian, unsigned integers are supported.
Parameters: |
|
---|
Typical users will create an IntegerField inside a BinaryStructure subclass definition:
class MyObject(BinaryStructure):
myfield = IntegerField(3, 6) # integer encoded in bytes 3, 4, 5
When you instantiate the subclass and access the field, its value will be big-endian unsigned integer encoded at that location in the structure. So with a file whose bytes 3, 4, and 5 are '\x00\x01\x04':
>>> o = MyObject('file.bin')
>>> o.myfield
260
EULcommon is a collection of common Python libraries in use at Emory University Libraries. It’s a bit miscellaneous: The libraries are collected together primarily to minimize proliferating many tiny projects. In future releases individual subpackages may be split out as they mature.
Customized decorators that enhance the default behavior of django.contrib.auth.decorators.permission_required().
The default behavior of django.contrib.auth.decorators.permission_required() for any user does not meet the required permission level is to redirect them to the login page– even if that user is already logged in. For more discussion of this behavior and current status in Django, see: http://code.djangoproject.com/ticket/4617
These decorators work the same way as the Django equivalents, with the added feature that if the user is already logged in and does not have the required permission, they will see 403 page instead of the login page.
The decorators should be used exactly the same as their django equivalents.
The code is based on the django snippet code at http://djangosnippets.org/snippets/254/
View decorator that checks to see if the user passes the specified test. See django.contrib.auth.decorators.user_passes_test().
Anonymous users will be redirected to login_url, while logged in users that fail the test will be given a 403 error. In the case of a 403, the function will render the 403.html template.
Decorator for views that checks whether a user has a particular permission enabled, redirecting to the login page or rendering a 403 as necessary.
See django.contrib.auth.decorators.permission_required().
Decorator for views that checks that the user passes the given test, redirecting to the log-in page if necessary. The test should be a callable that takes the user object and returns True if the user passes.
Returns special response to ajax calls instead of blindly redirecting.
To use with class methods instead of functions, use django.utils.decorators.method_decorator(). See http://docs.djangoproject.com/en/dev/releases/1.2/#user-passes-test-login-required-and-permission-required
Usage is the same as django.contrib.auth.decorators.user_passes_test():
@user_passes_test_with_ajax(lambda u: u.has_perm('polls.can_vote'), login_url='/loginpage/')
def my_view(request):
...
Decorator for views that checks that the user is logged in, redirecting to the log-in page if necessary, but returns a special response for ajax requests. See eulcommon.djangoextras.auth.decorators.user_passes_test_with_ajax().
Example usage:
@login_required_with_ajax()
def my_view(request):
...
Decorator for views that checks whether a user has a particular permission enabled, redirecting to the log-in page if necessary, but returns a special response for ajax requests. See eulcore.django.auth.decorators.user_passes_test_with_ajax().
Usage is the same as django.contrib.auth.decorators.permission_required()
@permission_required_with_ajax('polls.can_vote', login_url='/loginpage/')
def my_view(request):
...
Custom generic form fields for use with Django forms.
W3C date field that uses a W3CDateWidget for presentation and uses a simple regular expression to do basic validation on the input (but does not actually test that it is a valid date).
alias of W3CDateWidget
Multi-part date widget that generates three text input boxes for year, month, and day. Expects and generates dates in any of these W3C formats, depending on which fields are filled in: YYYY-MM-DD, YYYY-MM, or YYYY.
Generate and render a django.forms.widgets.TextInput for a single year, month, or day input.
If size is specified in the extra attributes, it will also be used to set the maximum length of the field.
Parameters: |
|
---|---|
Returns: | rendered HTML output for the text input |
Render the widget as HTML inputs for display on a form.
Parameters: |
|
---|---|
Returns: | HTML text with three inputs for year/month/day |
Generate a single value from multi-part form data. Constructs a W3C date based on values that are set, leaving out day and month if they are not present.
Parameters: |
|
---|---|
Returns: | string value |
A django.forms.ChoiceField whose choices are not static, but instead generated dynamically when referenced.
Parameters: | choices – callable; this will be called to generate choices each time they are referenced |
---|
alias of DynamicSelect
A Select widget whose choices are not static, but instead generated dynamically when referenced.
Parameters: | choices – callable; this will be called to generate choices each time they are referenced. |
---|
This module contains utilities for searching.
Takes a search string and parses it into a list of keywords and phrases.
Generate a dictionary of pages to show around the current page. Show 3 numbers on either side of the specified page, or more if close to end or beginning of available pages.
Parameters: |
|
---|---|
Return type: | dictionary; keys are page numbers, values are page labels |
Parse a string of search terms into keywords, phrases, and field/value pairs. Use quotes (” “) to designate phrases and field:value or field:”term term” to designated field value pairs. Returns a list of tuples where the first value is the field, or None for a word or phrase, second value is the keyword or phrase. Incomplete field value pairs will return a tuple with None for the value. For example:
parse_search_terms('grahame "frog and toad" title:willows')
Would result in:
[(None,'grahame'), (None, 'frog and toad'), ('title', 'willows')]
Django template tag to display pagination links for a paginated list of items.
Example use:
{% load search_utils %}
{% pagination_links paged_items show_pages %}
The following is a summary of changes and improvements to eulcommon. New features in each version should be listed, with any necessary information about installation or upgrade notes.