_images/img_1504.jpg

pywws

Python software for USB Wireless WeatherStations.

pywws is a collection of Python modules to read, store and process data from popular USB wireless weather stations such as Elecsa AstroTouch 6975, Watson W-8681, WH-1080PC, WH1080, WH1081, WH3080 etc. I assume any model that is supplied with the EasyWeather Windows software is compatible, but cannot guarantee this.

The software has been developed to run in a low power, low memory environment such as a router or Raspberry Pi. It can be used to create graphs and web pages showing recent weather readings, typically updated every hour. It can also send “live” data to services such as Weather Underground and post messages to Twitter.

The development version of pywws is hosted on GitHub.
“Snapshot” releases of pywws are available from the Python Package Index (PyPI).
Documentation is hosted on Read the Docs.
Documentation is available in the following languages (non-English versions may not be complete or up to date):

I have written this software to meet my needs, but have tried to make it adaptable to other people’s requirements. You may want to edit some or all of the modules, or write some new ones, to get it to do exactly what you want. One of the reasons for using Python is that it makes such alterations so easy. Don’t be afraid, just jump in and have a go.

Requirements

The software you’ll need to run pywws depends on what you plan to do with it. You’ll need Python 2.5 or later – Python 3 is partially supported, some functionality depends on libraries that have not yet been ported to Python 3.

For more detail, see Dependencies.

Installing and upgrading pywws

pywws can be installed directly from the Python Package Index (PyPI) using the pip command. See How to get started with pywws for full instructions.

Some new versions of pywws change what’s stored in the hourly, daily or monthly summary data files. These new versions are incompatible with processed data from earlier versions. The pywws.Reprocess script regenerates all the summary data. It should be run after any major upgrade.

Documentation

Documentation is included with pywws downloads, and is also available online. A good starting place is How to get started with pywws which describes in more detail how to install pywws.

If you have questions not answered in the documentation, please join the pywws Google mailing list / discussion group and ask there. Note that your first message to the group will not appear immediately – new posters have to be approved by a moderator, to prevent spam messages.

Contents

GNU General Public License

		    GNU GENERAL PUBLIC LICENSE
		       Version 2, June 1991

 Copyright (C) 1989, 1991 Free Software Foundation, Inc.
     51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
 Everyone is permitted to copy and distribute verbatim copies
 of this license document, but changing it is not allowed.

			    Preamble

  The licenses for most software are designed to take away your
freedom to share and change it.  By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users.  This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it.  (Some other Free Software Foundation software is covered by
the GNU Library General Public License instead.)  You can apply it to
your programs, too.

  When we speak of free software, we are referring to freedom, not
price.  Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.

  To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.

  For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have.  You must make sure that they, too, receive or can get the
source code.  And you must show them these terms so they know their
rights.

  We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.

  Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software.  If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.

  Finally, any free program is threatened constantly by software
patents.  We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary.  To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.

  The precise terms and conditions for copying, distribution and
modification follow.

		    GNU GENERAL PUBLIC LICENSE
   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

  0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License.  The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language.  (Hereinafter, translation is included without limitation in
the term "modification".)  Each licensee is addressed as "you".

Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope.  The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.

  1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.

You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.

  2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:

    a) You must cause the modified files to carry prominent notices
    stating that you changed the files and the date of any change.

    b) You must cause any work that you distribute or publish, that in
    whole or in part contains or is derived from the Program or any
    part thereof, to be licensed as a whole at no charge to all third
    parties under the terms of this License.

    c) If the modified program normally reads commands interactively
    when run, you must cause it, when started running for such
    interactive use in the most ordinary way, to print or display an
    announcement including an appropriate copyright notice and a
    notice that there is no warranty (or else, saying that you provide
    a warranty) and that users may redistribute the program under
    these conditions, and telling the user how to view a copy of this
    License.  (Exception: if the Program itself is interactive but
    does not normally print such an announcement, your work based on
    the Program is not required to print an announcement.)

These requirements apply to the modified work as a whole.  If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works.  But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.

Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.

In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.

  3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:

    a) Accompany it with the complete corresponding machine-readable
    source code, which must be distributed under the terms of Sections
    1 and 2 above on a medium customarily used for software interchange; or,

    b) Accompany it with a written offer, valid for at least three
    years, to give any third party, for a charge no more than your
    cost of physically performing source distribution, a complete
    machine-readable copy of the corresponding source code, to be
    distributed under the terms of Sections 1 and 2 above on a medium
    customarily used for software interchange; or,

    c) Accompany it with the information you received as to the offer
    to distribute corresponding source code.  (This alternative is
    allowed only for noncommercial distribution and only if you
    received the program in object code or executable form with such
    an offer, in accord with Subsection b above.)

The source code for a work means the preferred form of the work for
making modifications to it.  For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable.  However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.

If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.

  4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License.  Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.

  5. You are not required to accept this License, since you have not
signed it.  However, nothing else grants you permission to modify or
distribute the Program or its derivative works.  These actions are
prohibited by law if you do not accept this License.  Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.

  6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions.  You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.

  7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License.  If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all.  For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.

If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.

It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices.  Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.

This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.

  8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded.  In such case, this License incorporates
the limitation as if written in the body of this License.

  9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time.  Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.

Each version is given a distinguishing version number.  If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation.  If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.

  10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission.  For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this.  Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.

			    NO WARRANTY

  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.

  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.

		     END OF TERMS AND CONDITIONS

	    How to Apply These Terms to Your New Programs

  If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.

  To do so, attach the following notices to the program.  It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.

    <one line to give the program's name and a brief idea of what it does.>
    Copyright (C) <year>  <name of author>

    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program; if not, write to the Free Software
    Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA


Also add information on how to contact you by electronic and paper mail.

If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:

    Gnomovision version 69, Copyright (C) year  name of author
    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
    This is free software, and you are welcome to redistribute it
    under certain conditions; type `show c' for details.

The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License.  Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.

You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary.  Here is a sample; alter the names:

  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
  `Gnomovision' (which makes passes at compilers) written by James Hacker.

  <signature of Ty Coon>, 1 April 1989
  Ty Coon, President of Vice

This General Public License does not permit incorporating your program into
proprietary programs.  If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library.  If this is what you want to do, use the GNU Library General
Public License instead of this License.

Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

pywws Contributors

The copywright to pywws and its documentation is jointly held by the following contributors.

Developers
----------

Jim Easterbrook                 jim@jim-easterbrook.me.uk
x2q
3v1n0
Robin Kearney                   robin@kearney.co.uk
Rod Persky
Morten Høybye Frederiksen       morten@mfd-consult.dk
Simon Josefsson                 simon@josefsson.org
Matthew Hilton                  matthilton2005@gmail.com
Sabine Tobolka			            oe1yvw@gmail.com
Markus Birth                    markus@birth-online.de
Chris Ramsay                    chris@ramsay-family.net
Christian Benke                 benkokakao@gmail.com
Ian Wilkinson			null@sgtwilko.f9.co.uk

Translators
-----------

Edoardo                         edoardo69@hotmail.it
Jacques Desroches               metelsto@gmail.com
Sunshades                       joacim@ahlstrand.info
Johabu                          johabu96@yahoo.de
                                karte2@gmail.com
Kyle Gordon                     kyle@lodge.glasgownet.com>
Πέτρος                          nouvakis@sch.gr
Ramiro                          ramiro.sanchez@telefonica.net
Rick Sulman                     rick@sulman.org
Pyttsen                         weather@spacelab.se
Tech2304                        tech2304@gmail.com
Pablo Vera                      pablo.vera82@gmail.com
Contributing to pywws

If you would like to add a feature to pywws (or fix a problem with it) then please do. Open source software thrives when its users become active contributors. The process is quite simple:

  1. Join GitHub - it’s free.

  2. Fork the pywws repo - see Fork a Repo for help.

  3. Clone your fork to a computer you can use to develop your new feature.

  4. Use git to commit changes as you make them and push the changes to your fork of pywws.

    Please add a signed-off-by line to your commits which certify your developer certificate of origin (see below). For example, if your name is “John Smith”, and your email address is “jsmith@example.com”, just include the following line at the bottom of your commit messages:

    Signed-off-by: John Smith <jsmith@example.com>

    You should be able to do this automatically by using the -s option on your git commit commands.

  5. Add your name and email to the src/contributors/contributors.txt file. Don’t forget the -s option when you commit this change.

  6. Test your changes!

  7. When everything’s working as you expect, submit a Pull Request.

Developer Certificate of Origin

Including a signed-off-by line in your commits indicates that you certify the following:

Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
660 York Street, Suite 102,
San Francisco, CA 94110 USA

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

Clauses (a), (b) and (c) reassure pywws users that the project will remain open source well in to the future. Clause (d) reminds you that your contributions will be publicly available, and you do not have the right to withdraw them in future.


Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

Dependencies

The list of other software that pywws depends on looks frighteningly long at first glance. However, many of these packages won’t be needed by most users. What you need depends on what you want to do with pywws. Remember, it’s a “kit of parts” rather than a monolithic application.

Some of the requirements are Python packages that can be downloaded from the Python Package Index (PyPI). I recommend using pip to install these.

You should be able to install the remaining dependencies using your operating system’s package manager. This is a lot easier than downloading and compiling source files from the project websites. Note that some Linux distributions use different names for some of the packages, e.g. in Ubuntu, pyusb is called python-usb.

Note: some of these libraries may have their own dependencies that you may need to install. Follow the links to read more about each library’s requirements.

Essential
  • Python version 2.5 or higher

Python 3 is supported, but some things might not work properly. If you find a problem with Python 3, please send a message to the mailing list or submit a bug report on GitHub.

You will probably be able to install pip with your system’s package manager, where it may be called python-pip or python3-pip or something similar. If not, download and run the get-pip.py file from the pip web site. In either case you should immediately use pip to install the latest version of itself:

sudo pip install --upgrade pip

Make sure you install the correct Python version’s pip. If you want to install pywws for both Python 2 and Python 3 you will need pip2 and pip3.

This is a handy little module that provides information on your local time zone. It’s best installed with pip:

sudo pip install tzlocal
USB library

To retrieve data from a weather station pywws needs a python library that allows it to communicate via USB. There is a variety of USB libraries that can be used. Not all of them are available on all computing platforms, which may restrict your choice.

Mac OS X

On MacOS X the operating system’s generic hid driver “claims” the weather station, which makes it very difficult to use any other USB interface. Unfortunately, you will need to download and compile hidapi yourself.

  • hidapi
  • ctypes (your package manager may know it as python-ctypes)

If you can’t install ctypes then you can try the Cython interface to hidapi instead:

Other systems

Other systems use a Python interface to the libusb system library. There is a choice of interface and library version - install the latest that is available for your computer.

pip install libusb1

or

  • libusb version 1.x or version 0.1 (should be available from the package manager)
  • PyUSB version 1.0
pip install pyusb --pre

The --pre flag enables the installation of “pre release” versions, such as the current beta release (1.0.0b2) of pyusb.

If neither of these options works for you then you can use hidapi – see the Mac OS X instructions above.

Veranderd in versie 15.01.0.dev1265: added ability to use python-libusb1 interface.

Flexible timed tasks

The pywws.Tasks module can do tasks at particular times and/or dates. This requires the croniter library. (Simple hourly, daily or ‘live’ tasks don’t need this library.)

pip install croniter
Running as a daemon

The pywws.livelogdaemon program runs pywws live logging as a proper UNIX daemon process. It requires the python-daemon library:

pip install python-daemon
Graph drawing

The pywws.Plot module uses gnuplot to draw graphs. If you want to produce graphs of weather data, e.g. to include in a web page, you need to install the gnuplot application:

  • gnuplot v4.2 or higher (should be available from the package manager)

After installing gnuplot you should edit weather.ini (see weather.ini - configuration file format) and set the gnuplot version config item. Finding out the installed gnuplot version is easy:

gnuplot -V
Secure website uploading (sftp)

The pywws.Upload module can use “ftp over ssh” (sftp) to upload files to your web-site. Normal uploading just uses Python’s standard modules, but if you want to use sftp you need to install these two modules:

sudo pip install pycrypto paramiko
Twitter updates

The pywws.ToTwitter module can be used to send weather status messages to Twitter. Posting to Twitter requires these modules:

sudo pip install python-twitter oauth2

or

sudo pip install tweepy oauth2

Note that tweepy appears to be the less reliable of the two. If you have problems, e.g. with character encoding, try installing python-twitter instead.

Veranderd in versie 13.10_r1086: reenabled use of tweepy library as an alternative to python-twitter. python-oauth2 is still required by pywws.TwitterAuth.

Veranderd in versie 13.06_r1023: pywws previously used the tweepy library instead of python-twitter and python-oauth2.

MQTT

Nieuw in versie 14.12.0.dev1260.

The pywws.toservice module can be used to send weather data to an MQTT broker. This requires the paho-mqtt module:

sudo pip install paho-mqtt
To create new language translations

pywws can be configured to use languages other than English, as described in How to use pywws in another language. The Babel package is required to extract the strings to be translated and to compile the translation files.

sudo pip install babel

Copying files to or from Transifex requires the transifex-client package.

sudo pip install transifex-client

Translating the documentation using local files needs the sphinx-intl package.

sudo pip install sphinx-intl

Veranderd in versie 14.05.dev1209: pywws previously used the gettext package.

To ‘compile’ the documentation

The documentation of pywws is written in “ReStructured text”. A program called Sphinx is used to convert this easy to write format into HTML for use with a web browser. If you’d like to create a local copy of the documentation (so you don’t have to rely on the online version, or to test a translation you’re working on) you need to install Sphinx, version 1.3 or later.

sudo pip install sphinx

Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

Change Log

pywws - Python software for USB Wireless Weather Stations
http://github.com/jim-easterbrook/pywws
Copyright (C) 2008-18  pywws contributors

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.

Changes in v18.04.0:
    1/ Now works with broken weather stations that have stopped logging data.
       (Although there may be more gaps in the data.)
    2/ Cope better with missing wind direction data.
    3/ Use HTTPS for Weather Underground uploads.
    4/ Other minor bug fixes.

Changes in v17.11.0:
    1/ Allow MQTT service without password.
    2/ Allow SFTP uploads with public key.
    3/ Increase Twitter character limit from 140 to 280.
    4/ Various other bug fixes and minor improvements.

Changes in v16.12.0:
    1/ Added "candlestick" plot type.
    2/ Added cloud base calculation function.
    3/ Various other bug fixes and minor improvements.

Changes in v16.08.0:
    1/ Fix Python 2.5 incompatibilities.
    2/ Fix python-twitter v3 tweet length problem.

Changes in v16.07.1:
    1/ Further changes to handle UK Met Office server quirks.

Changes in v16.07.0:
    1/ Fix bug with UK Met Office uploads server change.
    2/ Allow user commands in wind roses.
    3/ Various other bug fixes and minor improvements.

Changes in v15.12.0:
    1/ Fix bug with Twitter messages being excessively truncated.
    2/ Improve handling of utf-8 encoded templates.
    3/ Improved plots and wind roses with 'pngcairo' "terminal".
    4/ Various bug fixes and minor improvements.

Changes in v15.11.0:
    1/ Add Russian translation of program text.
    2/ Improved documentation.
    3/ Various bug fixes and minor improvements.

Changes in v15.07.0:
    1/ Can include multiple media in Twitter messages.
    2/ Attempt to fix bug in wind rose axes labels.
    3/ Enable inclusion of time & date in wind rose title.
    4/ Various bug fixes and minor improvements.

Changes in v15.01.0:
    1/ Added 'MQTT' service.
    2/ Added another USB library option.
    3/ Improved Python 3 compatibility.
    4/ Various bug fixes and minor improvements.

Changes in v14.12.0:
    1/ Updated temperatur.nu and wetterarchiv.de service details to suit new
       APIs.

Changes in v14.06.1:
    1/ Revised version numbering scheme.
    2/ Compiled documentation no longer included in releases.
    3/ Can partially specify start & stop date/time in graphs, e.g. to start a
       plot at midnight, no matter when it is plotted.

Changes in v14.06:
    1/ Can now send images to Twitter.
    2/ Periodic tasks can be specified with a cron style syntax.
    3/ Added wind direction filter for use in graphs or user calibration
       modules.
    4/ Wind direction is now stored as a float. Old templates that use the
       wind_dir_text array will need updating, probably to use the
       winddir_text() function.
    5/ Started using "Transifex" to host translations. Changed tools and
       procedures to create new translations.
    6/ Improved USB hangup avoidance strategy for stations with large clock
       drift figures.
    7/ Various bug fixes and minor improvements.

Changes in v14.05:
    1/ Rearranged package layout, moving examples and documentation.
    2/ Added 'entry point' auto-generated commands for some modules.
    3/ Added verbose output option to pywws-version command.
    4/ Various bug fixes and minor improvements.

Changes in v14.03:
    1/ Extracts additional status from 'wind_dir' byte. You must run
       pywws-reprocess.py with the -u option after upgrading from any previous
       version.
    2/ Added Citizen Weather Observer Program to available 'services'.
    3/ Improved asynchronous upload task queuing.
    4/ Various bug fixes and minor improvements.

Changes in v14.02:
    1/ Improved time zone handling, including non whole hour time zones.
    2/ New 'frequent writes' config option.
    3/ Improved 'live log' sync, particularly with 3080 type stations.
    4/ Record recent memory pointer to improve detection of gaps in data.
    5/ Various bug fixes and minor improvements.

Changes in v13.12:
    1/ Changed API of user calibration module.
    2/ Can use python-twitter *or* tweepy library.
    3/ Added a script to run live logging as a UNIX daemon process.
    4/ Changed data store to use separate read and write caches.
    5/ Various bug fixes and minor improvements.

Changes in v13.10:
    1/ Changed Twitter library from tweepy to python-twitter.
    2/ Added ability to do uploads asynchronously.
    3/ Rearranged and improved documentation.
    4/ Various bug fixes and minor improvements.

Changes in v13.06:
    1/ Substantially rearranged directories, getting rid of 'code' and 'code3'.
    2/ Removed 'makefile' - everything is now done via 'setup.py'.
    3/ Removed 'RunModule.py' - use 'python -m pywws.module' now.
    4/ Separated storage of config (weather.ini) and status (status.ini).
    5/ Replaced toservice.py "rapid fire" mode with a separate config file for
       Weather Underground rapid fire.
    6/ Added 2 more low-level USB access modules.
    7/ Various bug fixes and minor improvements.

Changes in v13.03:
    1/ Added 'rain days' to monthly data. (Reprocess required when upgrading.)
    2/ Extended template syntax to include comments.
    3/ Added 'humidity index' function.
    4/ Added French translation of documentation.
    5/ Reduced frequency of saving data files.
    6/ Various bug fixes.

Changes in v12.12:
    1/ Added support for Python 3.
    2/ Added French documentation translation.
    3/ Used 'binary search' to speed up data access.
    4/ Various bug fixes.

Changes in v12.11:
    1/ Moved development from Google code to GitHub.
    2/ Made software attempt to avoid USB activity at times when it is assumed
       the weather station might be writing to its memory. This might solve
       the USB lockup problem, but it's too early to tell.

Changes in v12.10:
    1/ Added a 'winddir_text' function for use in templates.
    2/ Added <ytics> and <y2tics> options to graph plots.
    3/ Various bug fixes.

Changes in v12.07:
    1/ Added Open Weather Map to the services.
    2/ Fixed problem with Weather Underground uploads that started on 1st June.
    3/ Various bug fixes and software structure improvements.

Changes in v12.05:
    1/ Made 'fixed block' data available to template calculations.
    2/ Fixed buggy auto-detection of 3080 weather stations.
    3/ Added a function to generate the Zambretti forecast code letter.
    4/ Added a program to test USB communication reliablility.
    5/ Various bug fixes and software structure improvements.

Changes in v12.02:
    1/ Separated out low level USB communications to enable use of different
       libraries. Now works on recent versions of Mac OS.
    2/ Added humidity, pressure & wind data to summary data.
    3/ Merged Weather Underground and UK Met Office uploaders into one combined
       module. Added more 'service' uploaders.
    4/ Various bug fixes and software structure improvements.

Changes in v11.10:
    1/ Complete restructuring of documentation.
    2/ Added a user defined 'calibration' process.
    3/ Sets 'locale' according to language setting.
    4/ Added ability to upload to UK Met Office 'WOW'.
    5/ Various bug fixes and software structure improvements.
    6/ New language files: French, Danish.

Changes in v11.05:
    1/ Added support for '3080' family stations that have illuminance and
       UV sensors.
    2/ Broadened the range of tasks that can be done with 'live' data.
    3/ Various bug fixes and software structure improvements.

Changes in v11.02:
    1/ Various bug fixes and software structure improvements.
    2/ Improved wind direction averaging.
    3/ Added conversion functions for common things such as C to F.
    4/ Added a YoWindow module.
    5/ Improved Zambretti forecaster.

Changes in v10.12:
    1/ Various bug fixes and software structure improvements.
    2/ Added a 'goto' instruction to Template.py.
    3/ Added a 'Zambretti' forecast function to Template.py. This should
       be treated as an experiment, and not relied upon for accuracy.

Changes in v10.10:
    1/ Added 'catchup' mode to ToUnderground.py.
    2/ Created 'Tasks.py' to handle common tasks.
    3/ Made better use of Python's logger for info and error
       messages.
    4/ Changed over from 'python-twitter' to 'tweepy' for Twitter
       access. Twitter authorisation using OAuth now works.
    5/ Added 'LiveLog.py' live logging program.
    6/ Added 'SetWeatherStation.py' to do some configuration of weather
       station. No longer need EasyWeather to set logging interval!
    7/ Added 'Rapid Fire' ability to ToUnderground.py.
    8/ Added plain text versions of HTML documentation.
    9/ Many bug fixes and minor improvements.

Changes in v10.08:
    1/ Added internal temperature to daily and monthly summaries.
       Run Reprocess.py when upgrading from earlier versions.
    2/ Added 'prevdata' to Template.py. Allows calculations that
       compare values from different times.
    3/ Made 'pressure_offset' available to calculations in Plot.py
       and Template.py. This is only useful when using 'raw' data.
    4/ Improved synchronisation to weather station's clock when
       fetching stored data.

Changes in v10.06:
    1/ Improved localisation code.
    2/ Minor bug fixes.
    3/ Added Y axis label angle control to plots.

Changes in v10.04:
    1/ Changed version numbering to year.month.
    2/ Allowed "upload" to a local directory instead of ftp site.
    3/ Added "calc" option to text templates (Template.py).
    4/ Added -v / --verbose option to Hourly.py to allow silent operation.
    5/ Added internationalisation / localisation of some strings.
    6/ Made 'raw' data available to text templates.
    7/ Added ability to upload to Weather Underground.
    8/ Added dual axis and cumulative graph capability.

Changes in v0.9:
    1/ Added lowest daytime max and highest nighttime min temperatures
       to monthly data.
    2/ Added average temperature to daily and monthly data.
    3/ Added 'terminal' element to Plot.py templates for greater control
       over output appearance.
    4/ Added 'command' element to Plot.py templates for even more
       control, for advanced users.
    5/ Added secure upload option.
    6/ Minor speed improvements.

Changes in v0.8:
    1/ Added meteorological day end hour user preference
    2/ Attempts at Windows compatibility
    3/ Corrected decoding of wind data at speeds over 25.5 m/s
    4/ Improved speed with new data caching strategy

Changes in v0.7:
    1/ Several bug fixes, mostly around new weather stations with not
       much data
    2/ Added min & max temperature extremes to monthly data
    3/ Added template and workspace directory locations to weather.ini
    4/ Increased versatility of Plot.py with layout and title elements

Changes in v0.6:
    1/ Added monthly data
    2/ Changed 'pressure' to 'abs_pressure' or 'rel_pressure'

Changes in v0.5:
    1/ Small bug fixes.
    2/ Added start time to daily data
    3/ Replaced individual plot programs with XML "recipe" system

Changes in v0.4:
    1/ Can post brief messages to Twitter.
    2/ Now time zone aware. Uses UTC for data indexing and local time
       for graphs and text data files.

Changes in v0.3:
    1/ Now uses templates to generate text data
    2/ Added 28 day plot
    3/ Minor efficiency improvements
    4/ Improved documentation

Changes in v0.2:
    1/ Now uses Python csv library to read and write data
    2/ Creates hourly and daily summary files
    3/ Includes rain data in graphs


Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

User guides

Contents:

How to get started with pywws
Installation

First of all you need to install Python and a USB library (to allow Python to access the weather station). See Dependencies for more detail.

Create a directory for all your weather related files and change to it. For example (on a Linux or similar operating system):

mkdir ~/weather
cd ~/weather
Easy installation

The easiest way to install pywws is with the pip command:

sudo pip install pywws

Upgrading pywws is also a one line command:

sudo pip install -U pywws

Now you are ready to Test the weather station connection.

Download and extract

If you prefer not to use pip, or you want easy access to the pywws source files (e.g. to translate the documentation – see How to use pywws in another language), you can download and extract the files into your weather directory.

Visit http://pypi.python.org/pypi/pywws/ and download one of the .tar.gz or .zip files. Put it in your weather directory, then extract all the files, for example:

cd ~/weather
tar zxvf pywws-14.03.dev1178.tar.gz

or:

cd ~/weather
unzip pywws-14.03.dev1178.zip

This should create a directory (called pywws-14.03.dev1178 in this example) containing all the pywws source files. It is convenient to create a soft link to this awkwardly named directory:

cd ~/weather
ln -s pywws-14.03.dev1178 pywws

Upgrading a downloaded snapshot is the same process as the first installation. Download the .tar.gz or .zip file, extract its contents, then delete the soft link pointing to the old download and create one pointing to the new download. Once you are satisfied the new version is working OK you can delete the old download entirely.

Clone the repository

The PyPI files contain a snapshot release of the software - a new one is issued every few months. If you want to use the very latest version of pywws, e.g. to work on fixing a bug, you can get all the files you need from the GitHub repository. Install git and use it to clone the repos:

cd ~/weather
git clone https://github.com/jim-easterbrook/pywws.git

To upgrade you use git to pull any changes:

cd ~/weather/pywws
git pull
Install pywws

If you have downloaded or cloned the pywws source files, you need to use setup.py to install it:

cd ~/weather/pywws
python setup.py compile_catalog
python setup.py build
sudo python setup.py install

The python setup.py compile_catalog step is only needed if you want to use pywws in a language other than English. See Test the pywws translations for more detail.

Note to Python 3 users: this will generate and use Python 3 versions of the pywws software in ~/weather/pywws/build/lib.

Compile documentation (optional)

If you’d like to have a local copy of the pywws documentation (and have downloaded the source or cloned the repo) you can “compile” the English documentation. This requires the sphinx package:

cd ~/weather/pywws
python setup.py build_sphinx

Compiling the documentation in another language requires the additional step of compiling the translation files, which requires the sphinx-intl package. For example, to compile the French documentation:

cd ~/weather/pywws
sphinx-intl build --locale-dir src/pywws/lang -l fr
LANG=fr python setup.py build_sphinx

The compiled documentation should then be found at ~/weather/pywws/doc/html/index.html. See How to use pywws in another language for more detail.

Test the weather station connection

Now you’re ready to test your pywws installation. Connect the weather station (if not already connected) then run the pywws.TestWeatherStation module:

pywws-testweatherstation

If everything is working correctly, this should dump a load of numbers to the screen, for example:

0000 55 aa ff ff ff ff ff ff ff ff ff ff ff ff ff ff 05 20 01 51 11 00 00 00 81 00 00 0f 00 00 60 55
0020 ea 27 a0 27 00 00 00 00 00 00 00 10 10 12 13 45 41 23 c8 00 32 80 47 2d 2c 01 2c 81 5e 01 1e 80
0040 96 00 c8 80 a0 28 80 25 a0 28 80 25 03 36 00 05 6b 00 00 0a 00 f4 01 18 03 00 00 00 00 00 00 00
0060 00 00 4e 1c 63 0d 2f 01 73 00 7a 01 47 80 7a 01 47 80 e4 00 00 00 71 28 7f 25 bb 28 bd 25 eb 00
0080 0c 02 84 00 0e 01 e3 01 ab 03 dc 17 00 10 08 21 08 54 10 03 07 22 18 10 08 11 08 30 10 04 21 16
00a0 26 08 07 24 17 17 08 11 01 06 10 09 06 30 14 29 09 01 06 07 46 09 06 30 14 29 09 01 06 07 46 08
00c0 08 31 14 30 10 05 14 15 27 10 01 26 20 47 09 01 23 05 13 10 01 26 20 47 09 01 23 05 13 10 02 22
00e0 11 06 10 02 22 11 06 08 07 07 19 32 08 12 13 22 32 08 09 07 08 48 01 12 05 04 43 10 02 22 14 43

There are several reasons why this might not work. Most likely is a ‘permissions’ problem. This can be tested by running the command as root:

sudo pywws-testweatherstation

If this works then you may be able to allow your normal user account to access the weather station by setting up a ‘udev’ rule. The exact method may depend on your Linux version, but this is typically done by creating a file /etc/udev/rules.d/39-weather-station.rules containing the following:

ACTION!="add|change", GOTO="weatherstation_end"
SUBSYSTEM=="usb", ATTRS{idVendor}=="1941", ATTRS{idProduct}=="8021", GROUP="weatherstation"
LABEL="weatherstation_end"

Unplug and replug the station’s USB connection to force udev to apply the new rule. This allows any user in the group weatherstation to access the weather station. You need to create this group and add your normal user account to it – many Linux systems have a GUI for user and group management.

If you have any other problem, please ask for help on the pywws mailing list: http://groups.google.com/group/pywws

Set up your weather station

If you haven’t already done so, you should set your weather station to display the correct relative atmospheric pressure. (See the manual for details of how to do this.) pywws gets the offset between relative and absolute pressure from the station, so this should be set before using pywws.

You can get the correct relative pressure from your location by looking on the internet for weather reports from a nearby station, ideally an official one such as an airport. This is best done during calm weather when the pressure is almost constant over a large area.

Set the weather station logging interval

Your weather station probably left the factory with a 30 minute logging interval. This enables the station to store about 11 weeks of data. Most pywws users set up their computers to read data from the station every hour, or more often, and only need the station to store enough data to cover computer failures. The recommended interval is 5 minutes, which still allows 2 weeks of storage. Use pywws.SetWeatherStation to set the interval:

pywws-setweatherstation -r 5

Note that the weather station will not start using the new interval until the current 30 minute logging period is finished. This may cause “station is not logging data” errors when running pywws logging. If this happens you need to wait until the 30 minute logging period ends.

Log your weather station data

First, choose a directory to store all your weather station data. This will be written to quite frequently, so a disk drive is preferable to a flash memory stick or card, as these have a limited number of writes. In most cases your home directory is suitable, for example:

mkdir ~/weather/data

This directory is referred to elsewhere in the pywws documentation as your data directory.

Make sure your computer has the right date & time, and time zone, as these are used to label the weather station data. If you haven’t already done so, it’s worth setting up NTP to synchronise your computer to a ‘time server’.

The first time you run pywws.LogData it will create a configuration file in your data directory called ‘weather.ini’ and then stop. You need to edit the configuration file and change the line ws type = Unknown to ws type = 1080 or ws type = 3080. (If your weather station console displays solar illuminance you have a 3080 type, all others are 1080.) Then run pywws.LogData again. This may take several minutes, as it will copy all the data stored in your station’s memory. The pywws.LogData program has a ‘verbose’ option that increases the amount of messages it displays while running. This is useful when running it manually, for example:

python -m pywws.LogData -vvv ~/weather/data

(Replace ~/weather/data with your data directory, if it’s different.)

You should now have some data files you can look at. For example:

more ~/weather/data/raw/2012/2012-12/2012-12-16.txt

(Replace the year, month and day with ones that you have data for.)

Convert old EasyWeather data (optional)

If you had been running EasyWeather before deciding to use pywws, you can convert the data EasyWeather had logged to the pywws format. Find your EasyWeather.dat file and then convert it:

python -m pywws.EWtoPy EasyWeather.dat ~/weather/data
Set some configuration options

After running pywws.LogData there should be a configuration file in your data directory called ‘weather.ini’. Open this with a text editor. You should find something like the following:

[config]
ws type = 1080
logdata sync = 1
pressure offset = 9.4

You need to add a new entry in the [config] section called day end hour. This tells pywws what convention you want to use when calculating daily summary data. In the UK, the ‘meteorological day’ is usually from 09:00 to 09:00 GMT (10:00 to 10:00 BST during summer), so I use a day end hour value of 9. In other countries a value of 24 (or 0) might be more suitable. Note that the value is set in local winter time. You should not need to change it when daylight savings time is in effect.

After editing, your weather.ini file should look something like this:

[config]
ws type = 1080
logdata sync = 1
pressure offset = 9.4
day end hour = 9

You can also edit the pressure offset value to adjust how pywws calculates the relative (sea level) air pressure from the absolute value that the station measures. If you change the pressure offset or day end hour in future, you must update all your stored data by running pywws.Reprocess.

For more detail on the configuration file options, see weather.ini - configuration file format.

Veranderd in versie 13.10_r1082: made pressure offset a config item. Previously it was always read from the weather station.

Process the raw data

pywws.LogData just copies the raw data from the weather station. To do something useful with that data you probably need hourly, daily and monthly summaries. These are created by pywws.Process. For example:

python -m pywws.Process ~/weather/data

You should now have some processed files to look at:

more ~/weather/data/daily/2012/2012-12-16.txt

If you ever change your day end hour configuration setting, you will need to reprocess all your weather data. You can do this by running pywws.Reprocess:

python -m pywws.Reprocess ~/weather/data

You are now ready to set up regular or continuous logging, as described in How to set up ‘hourly’ logging with pywws or How to set up ‘live’ logging with pywws.

Read the documentation

You’re looking at it right now! The User guides section is probably the most useful bit to read first, but the Python programma´s en modules section has a lot more detail on the various pywws modules and commands.


Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

How to set up ‘hourly’ logging with pywws
Introduction

There are two quite different modes of operation with pywws. Traditionally pywws.Hourly would be run at regular intervals (usually an hour) from cron. This is suitable for fairly static websites, but more frequent updates can be useful for sites such as Weather Underground (http://www.wunderground.com/). The newer pywws.LiveLog program runs continuously and can upload data every 48 seconds.

Note that although this document (and the program name) refers to ‘hourly’ logging, you can run pywws.Hourly as often or as infrequently as you like, but don’t try to run it more often than double your logging interval. For example, if your logging interval is 10 minutes, don’t run pywws.Hourly more often than every 20 minutes.

Getting started

First of all, you need to install pywws and make sure it can get data from your weather station. See How to get started with pywws for details.

Try running pywws.Hourly from the command line, with a high level of verbosity so you can see what’s happening. Use the pywws-hourly command to run pywws.Hourly:

pywws-hourly -vvv ~/weather/data

Within five minutes (assuming you have set a 5 minute logging interval) you should see a ‘live_data new ptr’ message, followed by fetching any new data from the weather station and processing it.

Veranderd in versie 14.04.dev1194: the pywws-hourly command replaced scripts/pywws-hourly.py.

Configuring file locations

Open your weather.ini file with a text editor. You should have a [paths] section similar to the following (where xxx is your user name):

[paths]
work = /tmp/weather
templates = /home/xxx/weather/templates/
graph_templates = /home/xxx/weather/graph_templates/
local_files = /home/xxx/weather/results/

Edit these to suit your installation and preferences. work is an existing temporary directory used to store intermediate files, templates is the directory where you keep your text template files, graph_templates is the directory where you keep your graph template files and local_files is a directory where template output that is not uploaded to your web site is put. Don’t use the pywws example directories for your templates, as they will get over-written when you upgrade pywws.

Copy your text and graph templates to the appropriate directories. You may find some of the examples provided with pywws useful to get started. The pywws-version -v command should show you where the examples are on your computer.

Nieuw in versie 14.04.dev1194: the pywws-version command.

Configuring periodic tasks

In weather.ini you should have [logged], [hourly], [12 hourly] and [daily] sections similar to the following:

[logged]
services = []
plot = []
text = []

[hourly]
...

These specify what pywws.Hourly should do when it is run. Tasks in the [logged] section are done every time there is new logged data, tasks in the [hourly] section are done every hour, tasks in the [12 hourly] section are done twice daily and tasks in the [daily] section are done once per day.

The services entry is a list of online weather services to upload data to. The plot and text entries are lists of template files for plots and text files to be processed and, optionally, uploaded to your web site. Add the names of your template files and weather services to the appropriate entries, for example:

[logged]
services = ['underground', 'metoffice']
plot = []
text = []

[hourly]
services = []
plot = ['7days.png.xml', '24hrs.png.xml', 'rose_24hrs.png.xml']
text = [('tweet.txt', 'T'), '24hrs.txt', '6hrs.txt', '7days.txt']

[12 hourly]
services = []
plot = []
text = []

[daily]
services = []
plot = ['28days.png.xml']
text = [('forecast.txt', 'T'), 'allmonths.txt']

Note the use of the 'T' flag – this tells pywws to send the template result to Twitter instead of uploading it to your ftp site.

You can test that all these are working by removing the [last update] section from status.ini, then running pywws.Hourly again:

pywws-hourly -v ~/weather/data

Nieuw in versie 14.05.dev1211: [cron name] sections. If you need more flexibility in when tasks are done you can use [cron name] sections. See weather.ini - configuration file format for more detail.

Veranderd in versie 13.06_r1015: added the 'T' flag. Previously Twitter templates were listed separately in twitter entries in the [hourly] and other sections. The older syntax still works, but is deprecated.

Veranderd in versie 13.05_r1009: the last update information was previously stored in weather.ini, with last update entries in several sections.

Run as a cron job

Most UNIX/Linux systems have a ‘cron’ daemon that can run programs at certain times, even if you are not logged in to the computer. You edit a ‘crontab’ file to specify what to run and when to run it. For example, to run pywws.Hourly every hour, at zero minutes past the hour:

0 * * * *       pywws-hourly /home/xxx/weather/data

This might work, but if it didn’t you probably won’t get any error messages to tell you what went wrong. It’s much better to run a script that runs pywws.Hourly and then emails you any output it produces. Here’s the script I use:

#!/bin/sh
#
# weather station logger calling script

export PATH=$PATH:/usr/local/bin

if [ ! -d ~/weather/data/ ]; then
  exit
  fi

log=/var/log/log-weather

pywws-hourly -v ~/weather/data >$log 2>&1

# mail the log file
/home/jim/scripts/email-log.sh $log "weather log"

You’ll need to edit this quite a lot to suit your file locations and so on, but it gives some idea of what to do.


Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

How to set up ‘live’ logging with pywws
Introduction

There are two quite different modes of operation with pywws. Traditionally pywws.Hourly would be run at regular intervals (usually an hour) from cron. This is suitable for fairly static websites, but more frequent updates can be useful for sites such as Weather Underground (http://www.wunderground.com/). The newer pywws.LiveLog program runs continuously and can upload data every 48 seconds.

Getting started

First of all, you need to install pywws and make sure it can get data from your weather station. See How to get started with pywws for details.

If you have previously been using pywws.Hourly then disable your ‘cron’ job (or whatever else you use to run it) so it no longer runs. You should not run pywws.Hourly and pywws.LiveLog at the same time.

Try running pywws.LiveLog from the command line, with a high level of verbosity so you can see what’s happening. Use the pywws-livelog command to run pywws.LiveLog:

pywws-livelog -vvv ~/weather/data

Within five minutes (assuming you have set a 5 minute logging interval) you should see a ‘live_data new ptr’ message, followed by fetching any new data from the weather station and processing it. Let pywws.LiveLog run for a minute or two longer, then kill the process by typing ‘<Ctrl>C’.

Veranderd in versie 14.04.dev1194: the pywws-livelog command replaced scripts/pywws-livelog.py.

Configuring file locations

Open your weather.ini file with a text editor. You should have a [paths] section similar to the following (where xxx is your user name):

[paths]
work = /tmp/weather
templates = /home/xxx/weather/templates/
graph_templates = /home/xxx/weather/graph_templates/
local_files = /home/xxx/weather/results/

Edit these to suit your installation and preferences. work is an existing temporary directory used to store intermediate files, templates is the directory where you keep your text template files, graph_templates is the directory where you keep your graph template files and local_files is a directory where template output that is not uploaded to your web site is put. Don’t use the pywws example directories for your templates, as they will get over-written when you upgrade pywws.

Copy your text and graph templates to the appropriate directories. You may find some of the examples provided with pywws useful to get started. The pywws-version -v command should show you where the examples are on your computer.

Nieuw in versie 14.04.dev1194: the pywws-version command.

Configuring periodic tasks

In weather.ini you should have a [live] section similar to the following:

[live]
services = []
plot = []
text = []

This section specifies what pywws should do every time it gets a new reading from the weather station, i.e. every 48 seconds. The services entry is a list of online weather services to upload data to, e.g. ['underground_rf']. The plot and text entries are lists of template files for plots and text files to be processed and, optionally, uploaded to your web site. You should probably leave all of these blank except for services.

If you use YoWindow (http://yowindow.com/) you can add an entry to the [live] section to specify your YoWindow file, e.g.:

[live]
services = ['underground_rf']
text = [('yowindow.xml', 'L')]
...

Note the use of the 'L' flag – this tells pywws to copy the template result to your “local files” directory instead of uploading it to your ftp site.

If you don’t already have them, create four more sections in your weather.ini file: [logged], [hourly], [12 hourly] and [daily]. These sections should have similar entries to the [live] section, and specify what to do every time data is logged (5 to 30 minutes, depending on your logging interval), every hour, twice daily and once per day. Add the names of your template files to the appropriate entries, for example:

[logged]
services = ['underground', 'metoffice']
plot = []
text = []

[hourly]
services = []
plot = ['7days.png.xml', '24hrs.png.xml', 'rose_24hrs.png.xml']
text = [('tweet.txt', 'T'), '24hrs.txt', '6hrs.txt', '7days.txt']

[12 hourly]
services = []
plot = []
text = []

[daily]
services = []
plot = ['28days.png.xml']
text = [('forecast.txt', 'T'), 'allmonths.txt']

Note the use of the 'T' flag – this tells pywws to send the template result to Twitter instead of uploading it to your ftp site.

Nieuw in versie 14.05.dev1211: [cron name] sections. If you need more flexibility in when tasks are done you can use [cron name] sections. See weather.ini - configuration file format for more detail.

Veranderd in versie 13.06_r1015: added the 'T' flag. Previously Twitter templates were listed separately in twitter entries in the [hourly] and other sections. The older syntax still works, but is deprecated.

Veranderd in versie 13.05_r1013: added a 'yowindow.xml' template. Previously yowindow files were generated by a separate module, invoked by a yowindow entry in the [live] section. This older syntax still works, but is deprecated.

Asynchronous uploads

Nieuw in versie 13.09_r1057.

Uploading data to web sites or ‘services’ can sometimes take a long time, particularly if a site has gone off line and the upload times out. In normal operation pywws waits until all uploads have been processed before fetching any more data from the weather station. This can lead to data sometimes being missed.

The asynchronous item in the [config] section of weather.ini can be set to True to tell pywws.LiveLog to do these uploads in a separate thread.

Run in the background

Nieuw in versie 13.12.dev1118.

In order to have pywws.LiveLog carry on running after you finish using your computer it needs to be run as a “background job”. On most Linux / UNIX systems you can do this by putting an ampersand (‘&’) at the end of the command line. Running a job in the background like this doesn’t always work as expected: the job may suspend when you log out. It’s much better to run as a proper UNIX ‘daemon’ process.

The pywws.livelogdaemon program does this, if you have the python-daemon library installed:

pywws-livelog-daemon -v ~/weather/data ~/weather/data/pywws.log start

Note that the log file is a required parameter, not an option.

Automatic restarting

There are various ways of configuring a Linux system to start a program when the machine boots up. Typically these involve putting a file in /etc/init.d/, which requires root privileges. A slightly harder problem is ensuring a program restarts if it crashes. My solution to both problems is to run the following script from cron, several times an hour.

#!/bin/sh

export PATH=$PATH:/usr/local/bin

# exit if NTP hasn't set computer clock
[ `ntpdc -c sysinfo | awk '/stratum:/ {print $2}'` -ge 10 ] && exit

pidfile=/var/run/pywws.pid
datadir=/home/jim/weather/data
logfile=$datadir/live_logger.log

# exit if process is running
[ -f $pidfile ] && kill -0 `cat $pidfile` && exit

# email last few lines of the logfile to see why it died
if [ -f $logfile ]; then
  log=/tmp/log-weather
  tail -40 $logfile >$log
  /home/jim/scripts/email-log.sh $log "weather log"
  rm $log
  fi

# restart process
pywws-livelog-daemon -v -p $pidfile $datadir $logfile start

The process id of the daemon is stored in pidfile. If the process is running, the script does nothing. If the process has crashed, it emails the last 40 lines of the log file to me (using a script that creates a message and passes it to sendmail) and then restarts pywws.livelogdaemon. You’ll need to edit this quite a lot to suit your file locations and so on, but it gives some idea of what to do.


Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

How to integrate pywws with various weather services

This guide gives brief instructions on how to use pywws with some other weather services and software. It is not comprehensive, and some services (such as Twitter) are covered in more detail elsewhere.

YoWindow

YoWindow is a weather display widget that can display data from an internet source, or from your weather station. To display data from your station pywws needs to write to a local file, typically every 48 seconds when new data is received. This is easy to do:

  1. Stop all pywws software

  2. Copy the yowindow.xml example template to your text template directory.

  3. If you haven’t already done so, edit weather.ini and set the local_files entry in the [paths] section to a suitable directory for your yowindow file.

  4. Add the yowindow template to the [live] tasks in weather.ini. Set its flags to 'L' so the result is copied to your local directory instead of being uploaded to an ftp site:

    [live]
    text = [('yowindow.xml', 'L')]
    
  5. Restart pywws live logging.

You can check the file is being updated every 48 seconds by using more or cat to dump it to the screen.

Finally configure yowindow to use this file. See http://yowindow.com/pws_setup.php for instructions on how to do this.

Twitter

See How to configure pywws to post messages to Twitter for full instructions.

Other “services”

The remaining weather service uploads are handled by the pywws.toservice module. See the module’s documentation for general configuration options. The following subsections give further information about some of the available services.

Citizen Weather Observer Program

Nieuw in versie 14.02.dev1156.

  • Web site: http://www.wxqa.com/

  • Create account: http://www.wxqa.com/SIGN-UP.html

  • API: http://www.wxqa.com/faq.html

  • Example weather.ini section:

    [cwop]
    designator = EW9999
    latitude = 5130.06N
    longitude = 00008.52E
    template = default
    
    [logged]
    services = ['cwop', 'underground']
    
    [live]
    services = ['cwop', 'underground_rf']
    

    or, for radio hams:

    [cwop]
    designator = G4XXX
    passcode = xxxxxx
    latitude = 5130.06N
    longitude = 00008.52E
    template = default
    
    [logged]
    services = ['cwop_ham', 'underground']
    
    [live]
    services = ['cwop_ham', 'underground_rf']
    

Note that the latitude and longitude must be in “LORAN” format and leading zeros are required. See question 3 in the CWOP FAQ for more information.

Licensed radio hams use their callsign as the designator and need a passcode. They should use the service name cwop_ham instead of cwop when running pywws.toservice directly and in the weather.ini services entries. (The same [cwop] config section is used for both.)

CWOP uploads are rate-limited by pywws, so you can safely add it to both the [live] and [logged] sections in weather.ini.

The CWOP/APRS uploader is based on code by Marco Trevisan <mail@3v1n0.net>.

MQTT

Nieuw in versie 14.12.0.dev1260.

MQTT is a “message broker” system, typically running on localhost or another computer in your home network. Use of MQTT with pywws requires an additional library. See Dependencies - MQTT for details.

  • MQTT: http://mqtt.org/

  • Mosquitto (a lightweight broker): http://mosquitto.org/

  • Example weather.ini section:

    [mqtt]
    topic = /weather/pywws
    hostname = localhost
    port = 1883
    client_id = pywws
    retain = True
    auth = False
    user = unknown
    password = unknown
    template = default
    multi_topic = False
    
    [logged]
    services = ['mqtt', 'underground']
    

pywws will publish a JSON string of the data specified in the mqtt_template_1080.txt file. This data will be published to the broker running on hostname, with the port number specified. (An IP address can be used instead of a host name.) client_id is a note of who published the data to the topic. topic can be any string value, this needs to be the topic that a subscriber is aware of.

retain is a boolean and should be set to True or False (or left at the default unknown). If set to True this will flag the message sent to the broker to be retained. Otherwise the broker discards the message if no client is subscribing to this topic. This allows clients to get an immediate response when they subscribe to a topic, without having to wait until the next message is published.

auth, user and password can be used for MQTT authentication.

multi_topic is a boolean and should be set to True or False. If set to True pywws will also publish all the data each as separate subtopics of the configured topic; i.e., with the topic set to /weather/pywws pywws will also publish the outside temperature to /weather/pywws/temp_out and the inside temperature to /weather/pywws/temp_in.

If these aren’t obvious to you it’s worth doing a bit of reading around MQTT. It’s a great lightweight messaging system from IBM, recently made more popular when Facebook published information on their use of it.

This has been tested with the Mosquitto Open Source MQTT broker, running on a Raspberry Pi (Raspian OS). TLS (mqtt data encryption) is not yet implemented.

Thanks to Matt Thompson for writing the MQTT code and to Robin Kearney for adding the retain and auth options.

UK Met Office
Open Weather Map

When choosing a user name you should avoid spaces (and probably non-ascii characters as well). Having a space in your user name causes strange “internal server error” responses from the server.

The default behaviour is to use your user name to identify the weather station. However, it’s possible for a user to have more than one weather station, so there is an optional name parameter in the API that can be used to identify the station. This appears as id in weather.ini. Make sure you choose a name that is not already in use.

PWS Weather
temperatur.nu
  • Web site: http://www.temperatur.nu/

  • Example weather.ini section:

    [temperaturnu]
    hash = ???
    template = default
    
    [logged]
    services = ['temperaturnu', 'underground']
    

You receive the hash value from the temperatur.nu admins during sign up. It looks like “d3b07384d113edec49eaa6238ad5ff00”.

Weather Underground
Weather Underground “RapidFire” updates

Weather Underground has a second upload URL for real time updates as little as 2.5 seconds apart. If you run pywws in ‘live logging’ mode (see How to set up ‘live’ logging with pywws) you can use this to send updates every 48 seconds, by adding ‘underground_rf’ to the [live] tasks section in weather.ini:

[underground]
station = ABCDEFGH1
password = xxxxxxx
template = default

[live]
services = ['underground_rf']

[logged]
services = ['underground', 'metoffice']

Make sure you still have an ‘underground’ service in [logged] or [hourly]. This will ensure that ‘catchup’ records are sent to fill in any gaps if your station goes offline for some reason.

wetter.com
Custom Request Headers

The pywws.toservice module does support the injection of one or more custom request headers for special cases where you want to integrate with a service that, for example, requires you to pass an authentication key header along with each request, such as x-api-key.

These headers can be added to your a_service.ini file in the format of key value pairs:

[config]
url         = https://my-aws-api-gw.execute-api.eu-west-1.amazonaws.com/test/station
catchup             = 100
interval    = 0
use get             = True
result              = []
auth_type   = None
http_headers        = [('x-api-key', 'my-api-key'), ('x-some-header', 'value')]

Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

How to configure pywws to post messages to Twitter
Install dependencies

Posting to Twitter requires some extra software. See Dependencies - Twitter updates.

Create a Twitter account

You could post weather updates to your ‘normal’ Twitter account, but I think it’s better to have a separate account just for weather reports. This could be useful to someone who lives in your area, but doesn’t want to know what you had for breakfast.

Sta pywws toe berichten te plaatsen op je Twitter account.

If you run pywws on a low power device such as a router, you may find it easier to run this authorisation step on another computer, as long as it has python-oauth2 installed. Use an empty ‘data’ directory – a weather.ini file will be created whose contents can be copied into your real weather.ini file using any text editor.

Make sure no other pywws software is running, then run TwitterAuth:

python -m pywws.TwitterAuth ~/weather/data

(Replace ~/weather/data with your data directory.)

This will open a web browser window (or give you a URL to copy to your web browser) where you can log in to your Twitter account and authorise pywws to post. Your web browser will then show a 7 digit number which you need to copy to the TwitterAuth program. If successful, your weather.ini file will now have a [twitter] section with secret and key entries. (Don’t disclose these to anyone else.)

Add location data (optional)

Edit your weather.ini file and add latitude and longitude entries to the [twitter] section. For example:

[twitter]
secret = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
latitude = 51.501
longitude = -0.142
Create a template

Twitter messages are generated using a template, just like creating files to upload to a website. Copy the example template ‘tweet.txt’ to your template directory, then test it:

python -m pywws.Template ~/weather/data ~/weather/templates/tweet.txt tweet.txt
cat tweet.txt

(Replace ~/weather/data and ~/weather/templates with your data and template directories.) If you need to change the template (e.g. to change the units or language used) you can edit it now or later.

Post your first weather Tweet

Now everything is prepared for ToTwitter to be run:

python -m pywws.ToTwitter ~/weather/data tweet.txt

If this works, your new Twitter account will have posted its first weather report. (You should delete the tweet.txt file now.)

Add Twitter updates to your hourly tasks

Edit your weather.ini file and edit the [hourly] section. For example:

[hourly]
services = []
plot = ['7days.png.xml', '24hrs.png.xml', 'rose_12hrs.png.xml']
text = [('tweet.txt', 'T'), '24hrs.txt', '6hrs.txt', '7days.txt']

Note the use of the 'T' flag – this tells pywws to tweet the template result instead of uploading it to your ftp site.

You could change the [logged], [12 hourly] or [daily] sections instead, but I think [hourly] is most appropriate for Twitter updates.

Veranderd in versie 13.06_r1015: added the 'T' flag. Previously Twitter templates were listed separately in twitter entries in the [hourly] and other sections. The older syntax still works, but is deprecated.

Include an image in your tweet

Nieuw in versie 14.05.dev1216.

You can add up to four images to your tweets by specifying the image file locations in the tweet template. Make the first line of the tweet media path where path is the absolute location of the file. Repeat for any additional image files. The “tweet_media.txt” example template shows how to do this.

The image could be from a web cam, or for a weather forecast it could be an icon representing the forecast. To add a weather graph you need to make sure the graph is drawn before the tweet is sent. I do this by using two [cron xxx] sections in weather.ini:

[cron prehourly]
format = 59 * * * *
services = []
plot = [('tweet.png.xml', 'L')]
text = []

[cron hourly]
format = 0 * * * *
services = []
plot = ['7days.png.xml', '24hrs.png.xml', 'rose_12hrs.png.xml']
text = [('tweet_media.txt', 'T'), '24hrs.txt', '6hrs.txt', '7days.txt']

Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

How to use pywws in another language
Introduction

Some parts of pywws can be configured to use your local language instead of British English. This requires an appropriate language file which contains translations of the various strings used in pywws. The pywws project relies on users to provide these translations.

The pywws documentation can also be translated into other languages. This is a lot more work, but could be very useful to potential users who do not read English very well.

Using existing language files
Program strings

There may already be a pywws translation for your preferred language. First you need to choose the appropriate two-letter code from the list at http://www.w3schools.com/tags/ref_language_codes.asp. For example, fr is the code for French. Now use the pywws.Localisation module to do a quick test:

python -m pywws.Localisation -t fr

This should produce output something like this:

Locale changed from (None, None) to ('fr_FR', 'UTF-8')
Translation set OK
Locale
  decimal point: 23,2
  date & time: lundi, 17 décembre (17/12/2012 16:00:48)
Translations
  'NNW' => 'NNO'
  'rising very rapidly' => 'en hausse très rapide'
  'Rain at times, very unsettled' => 'Quelques précipitations, très perturbé'

This shows that pywws is already able to generate French output, and that your installation is correctly configured. Now you can edit the language entry in your weather.ini file to use your language code.

If the above test shows no translations into your language then you need to create a new language file, as described below.

Text encodings

The pywws default text encoding is ISO-8859-1, also known as Latin-1. This is suitable for several western European languages but not for some others. If you encounter problems you may need to use a different encoding. See the documentation of pywws.Template and pywws.Plot for more details.

Documentation

If you have downloaded the pywws source files, or cloned the GitHub repository (see how to get started with pywws), you can compile a non-English copy of the documentation. This requires the Sphinx package, see dependencies.

First delete the old documentation (if it exists) and then recompile using your language:

cd ~/weather/pywws
rm -rf doc
LANG=fr python setup.py build_sphinx

Note that the build_sphinx command doesn’t have a --locale (or -l) option, so the language is set by a temporary environment variable.

You can view the translated documentation by using a web browser to read the file ~/weather/pywws/doc/html/index.html.

Writing new language files

There are two ways to write new language files (or update existing ones) – use the Transifex online system or use local files. Transifex is preferred as it allows several people to work on the same language, and makes your work instantly available to others.

To test your translation you will need to have downloaded the pywws source files, or cloned the GitHub repository (see how to get started with pywws). You will also need to install the Babel package, see dependencies.

Using Transifex

If you’d like to use Transifex, please go to the pywws Transifex project, click on “help translate pywws” and create a free account.

Visit the pywws project page on Transifex and click on your language, then click on the “resource” you want to translate. (pywws contains the program strings used when running pywws, the others contain strings from the pywws documentation.) This opens a dialog where you can choose to translate the strings online. Please read Notes for translators before you start.

When you have finished translating you should use the transifex-client program (see dependencies) to download files for testing. For example, this command downloads any updated files for the French language:

cd ~/weather/pywws
tx pull -l fr

Now you are ready to Test the pywws translations.

Using local files

If you prefer not to use the Transifex web site you can edit language files on your own computer. This is done in two stages, as follows.

Extract source strings

Program messages are extracted using the Babel package:

cd ~/weather/pywws
mkdir -p build/gettext
python setup.py extract_messages

This creates the file build/gettext/pywws.pot. This is a “portable object template” file that contains the English language strings to be translated.

The documentation strings are extracted using the Sphinx package:

cd ~/weather/pywws
python setup.py extract_messages_doc

This creates several .pot files in the build/gettext/ directory.

Create language files

The sphinx-intl command is used to convert the .pot files to language specific .po files:

cd ~/weather/pywws
sphinx-intl update --locale-dir src/pywws/lang -p build/gettext -l fr

Now you can open the .po files in src/pywws/lang/fr/LC_MESSAGES/ with your favourite text editor and start filling in the empty msgstr strings with your translation of the corresponding msgid string. Please read Notes for translators before you start.

Test the pywws translations

The Babel package is used to compile program strings:

python setup.py compile_catalog --locale fr

(Replace fr with the code for the language you are testing.)

After compilation you can test the translation:

python setup.py build
sudo python setup.py install
python -m pywws.Localisation -t fr

Sphinx is used to build the translated documentation:

cd ~/weather/pywws
rm -rf doc
LANG=fr python setup.py build_sphinx

You can view the translated documentation by using a web browser to read the file ~/weather/pywws/doc/html/index.html.

Notes for translators

The pywws program strings (pywws.po) are quite simple. They comprise simple weather forecasts (“Fine weather”), air pressure changes (“rising quickly”) and the 16 points of the compass (“NNE”). Leave the “(%Z)” in “Time (%Z)” unchanged and make sure your translation’s punctuation matches the original.

The other files contain strings from the pywws documentation. These are in reStructuredText. This is mostly plain text, but uses characters such as backquotes (`), colons (:) and asterisks (*) for special purposes. You need to take care to preserve this special punctuation. Do not translate program source, computer instructions and cross-references like these:

`pip <http://www.pip-installer.org/>`_
:py:class:`datetime.datetime`
:obj:`ParamStore <pywws.DataStore.ParamStore>`\\ (root_dir, file_name)
pywws.Forecast
``pywws-livelog``

Translating all of the pywws documentation is a lot of work. However, when the documentation is “compiled” any untranslated strings revert to their English original. This means that a partial translation could still be useful – I suggest starting with the documentation front page, index.po.

Send Jim the translation

I’m sure you would like others to benefit from the work you’ve done in translating pywws. If you’ve been using Transifex then please send me an email (jim@jim-easterbrook.me.uk) to let me know there’s a new translation available. Otherwise, please email me any .po files you create.


Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

weather.ini - configuration file format

Nearly all configuration of pywws is via a single file in the data directory: weather.ini. This file has a structure similar to that of Microsoft Windows INI files. It is divided into “sections”, each of which has a number of “name = value” entries. The order in which sections appear is not important.

Any plain text editor can be used to do edit the file. (Don’t try to edit it while any other pywws software is running.) In many cases pywws will initialise the entries to sensible values.

Another file, status.ini, is used to store some information that pywws uses internally. It is described at the end of this document. In normal use you should not need to edit it.

The following sections are currently in use:

  • config: miscellaneous system configuration.
  • paths: directories in which templates etc. are stored.
  • live: tasks to be done every 48 seconds.
  • logged: tasks to be done every time the station logs a data record.
  • cron: tasks to be done at a particular time or date.
  • hourly: tasks to be done every hour.
  • 12 hourly: tasks to be done every 12 hours.
  • daily: tasks to be done every day.
  • ftp: configuration of uploading to a website.
  • twitter: configuration of posting to Twitter.
  • underground, metoffice, temperaturnu etc: configuration of posting to ‘services’.
config: miscellaneous system configuration
[config]
ws type = 1080
day end hour = 21
pressure offset = 9.4
gnuplot encoding = iso_8859_1
template encoding = iso-8859-1
language = en
logdata sync = 1
rain day threshold = 0.2
asynchronous = False
usb activity margin = 3.0
gnuplot version = 4.2
frequent writes = False

ws type is the “class” of weather station. It should be set to 1080 for most weather stations, or 3080 if your station console displays solar illuminance.

day end hour is the end of the “meteorological day”, in local time without daylight savings time. Typical values are 21, 9, or 24. You must update all your stored data by running pywws.Reprocess after you change this value.

pressure offset is the difference between absolute and relative (sea level) air pressure. The initial value is copied from the weather station, assuming you have set it up to display the correct relative pressure, but you can adjust the value in weather.ini to calibrate your station. You must update all your stored data by running pywws.Reprocess after you change this value.

Veranderd in versie 13.10_r1082: made pressure offset a config item. Previously it was always read from the weather station.

gnuplot encoding is the text encoding used when plotting graphs. The default value of iso_8859_1 allows the degree symbol, which is useful in a weather application! Other values might be needed if your language includes accented characters. The possible values depend on your gnuplot installation so some experimentation may be needed.

template encoding is the text encoding used for templates. The default value is iso-8859-1, which is the encoding used in the example templates. If you create templates with a different character set, you should change this value to match your templates.

language is used to localise pywws. It’s optional, as pywws usually uses the computer’s default language as set by the LANG environment variable. The available languages are those in the translations subdirectory of your pywws installation. If you set any other language, pywws will fall back to using English.

logdata sync sets the quality of synchronisation used by pywws.LogData. Set it to 0 for fast & inaccurate or 1 for slower but precise.

rain day threshold is the amount of rain (in mm) that has to fall in one day for it to qualify as a rainy day in the monthly summary data. You must update all your stored data by running pywws.Reprocess after you change this value.

Nieuw in versie 13.09_r1057: asynchrouous controls the use of a separate upload thread in pywws.LiveLog.

Nieuw in versie 13.10_r1094: usb activity margin controls the algorithm that avoids the “USB lockup” problem that affects some stations. It sets the number of seconds either side of expected station activity (receiving a reading from outside or logging a reading) that pywws does not get data from the station. If your station is not affected by the USB lockup problem you can set usb activity margin to 0.0.

Nieuw in versie 13.11_r1102: gnuplot version tells pywws.Plot and pywws.WindRose what version of gnuplot is installed on your computer. This allows them to use version-specific features to give improved plot quality.

Nieuw in versie 14.01_r1133: frequent writes tells pywws.Tasks to save weather data and status to file every time there is new logged data. The default is to save the files every hour, to reduce “wear” on solid state memory such as the SD cards used with Raspberry Pi computers. If your weather data directory is stored on a conventional disc drive you can set frequent writes to True.

paths: directories in which templates etc. are stored
[paths]
templates = /home/$USER/weather/templates/
graph_templates = /home/$USER/weather/graph_templates/
user_calib = /home/$USER/weather/modules/usercalib
work = /tmp/weather
local_files = /home/$USER/weather/results/

These entries specify where your text templates and graph templates are stored, where temporary files should be created, where template output (that is not uploaded) should be put, and (if you have one) the location of your calibration module.

live: tasks to be done every 48 seconds
[live]
services = ['underground_rf']
text = [('yowindow.xml', 'L')]
plot = []

This section specifies tasks that are to be carried out for every data sample during ‘live logging’, i.e. every 48 seconds.

services is a list of ‘services’ to upload data to. Each one listed must have a configuration file in pywws/services/. See ../api/pywws.toservice for more detail. pywws will automatically limit the frequency of service uploads according to each service’s specification.

text and plot are lists of text and plot templates to be processed and, optionally, uploaded to your website.

Veranderd in versie 13.05_r1013: added a 'yowindow.xml' template. Previously yowindow files were generated by a separate module, invoked by a yowindow entry in the [live] section. This older syntax still works, but is deprecated.

logged: tasks to be done every time the station logs a data record
[logged]
services = ['underground', 'metoffice']
text = []
plot = []

This section specifies tasks that are to be carried out every time a data record is logged when ‘live logging’ or every time an hourly cron job is run.

services is a list of ‘services’ to upload data to. Each one listed must have a configuration file in pywws/services/. See ../api/pywws.toservice for more detail.

text and plot are lists of text and plot templates to be processed and, optionally, uploaded to your website.

cron: tasks to be done at a particular time or date

Nieuw in versie 14.05.dev1211.

[cron prehourly]
format = 59 * * * *
plot = [('tweet.png.xml', 'L')]
services = []
text = []

[cron hourly]
format = 0 * * * *
plot = ['7days.png.xml', '2014.png.xml', '24hrs.png.xml', 'rose_12hrs.png.xml']
text = [('tweet.txt', 'T'), '24hrs.txt', '6hrs.txt', '7days.txt', '2014.txt']
services = []

[cron daily 9]
format = 0 9 * * *
plot = ['28days.png.xml']
text = [('forecast.txt', 'T'), 'forecast_9am.txt', 'forecast_week.txt']
services = []

[cron daily 21]
format = 0 21 * * *
text = ['forecast_9am.txt']
services = []
plot = []

[cron weekly]
format = 0 9 * * 6
plot = ['2008.png.xml', '2009.png.xml', '2010.png.xml', '2011.png.xml',
        '2012.png.xml', '2013.png.xml']
text = ['2008.txt', '2009.txt', '2010.txt', '2011.txt', '2012.txt', '2013.txt']
services = []

[cron name] sections provide a very flexible way to specify tasks to be done at a particular time and/or date. name can be anything you like, but each [cron name] section must have a unique name.

To use [cron name] sections you need to install the “croniter” package. See Dependencies for more detail.

format specifies when the tasks should be done (in local time), in the usual crontab format. (See man 5 crontab on any Linux computer.) Processing is not done exactly on the minute, but when the next live or logged data arrives.

hourly: tasks to be done every hour
[hourly]
services = []
text = [('tweet.txt', 'T'), '24hrs.txt', '6hrs.txt', '7days.txt', 'feed_hourly.xml']
plot = ['7days.png.xml', '24hrs.png.xml', 'rose_12hrs.png.xml']

This section specifies tasks that are to be carried out every hour when ‘live logging’ or running an hourly cron job.

services is a list of ‘services’ to upload data to. Each one listed must have a configuration file in pywws/services/. See ../api/pywws.toservice for more detail.

text and plot are lists of text and plot templates to be processed and, optionally, uploaded to your website.

Veranderd in versie 13.06_r1015: added the 'T' flag. Previously Twitter templates were listed separately in twitter entries in the [hourly] and other sections. The older syntax still works, but is deprecated.

12 hourly: tasks to be done every 12 hours
[12 hourly]
services = []
text = []
plot = []

This section specifies tasks that are to be carried out every 12 hours when ‘live logging’ or running an hourly cron job. Use it for things that don’t change very often, such as monthly graphs. The tasks are done at your day end hour, and 12 hours later.

services is a list of ‘services’ to upload data to. Each one listed must have a configuration file in pywws/services/. See ../api/pywws.toservice for more detail.

text and plot are lists of text and plot templates to be processed and, optionally, uploaded to your website.

daily: tasks to be done every 24 hours
[daily]
services = []
text = ['feed_daily.xml']
plot = ['2008.png.xml', '2009.png.xml', '2010.png.xml', '28days.png.xml']

This section specifies tasks that are to be carried out every day when ‘live logging’ or running an hourly cron job. Use it for things that don’t change very often, such as monthly or yearly graphs. The tasks are done at your day end hour.

services is a list of ‘services’ to upload data to. Each one listed must have a configuration file in pywws/services/. See ../api/pywws.toservice for more detail.

text and plot are lists of text and plot templates to be processed and, optionally, uploaded to your website.

ftp: configuration of uploading to a website
[ftp]
local site = False
secure = False
site = ftp.your_isp.co.uk
user = username
password = userpassword
directory = public_html/weather/data/
port = 21

These entries provide details of your website (or local directory) where processed text files and graph images should be transferred to.

local site specifies whether the files should be copied to a local directory or sent to a remote site. You may want to set this if you run your web server on the same machine as you are running pywws on.

secure specifies whether to transfer files using SFTP (secure FTP) instead of the more common FTP. Your web site provider should be able to tell you if you can use SFTP. Note that you may need to change the port value when you change to or from secure mode.

site is the web address of the FTP site to transfer files to.

user and password are the FTP site login details. Your web site provider should have provided them to you.

privkey is the path to a private SSH-key. For SFTP (secure FTP) this can be used for authentication instead of a password, which offers additional benefits in terms of security. When this is used the password-parameter can be left empty.

directory specifies where on the FTP site (or local file system) the files should be stored. Note that you may have to experiment with this a bit - you might need a ‘/’ character at the start of the path.

Nieuw in versie 13.12.dev1120: port specifies the port number to use. Default value is 21 for FTP, 22 for SFTP. Your web site provider may tell you to use a different port number.

twitter: configuration of posting to Twitter
[twitter]
secret = longstringofrandomcharacters
key = evenlongerstringofrandomcharacters
latitude = 51.365
longitude = -0.251

secret and key are authentication data provided by Twitter. To set them, run pywws.TwitterAuth.

latitude and longitude are optional location data. If you include them then your weather station tweets will have location information so users can see where your weather station is. It might also enable people to find your weather station tweets if they search by location.

underground, metoffice, temperaturnu etc: configuration of posting to ‘services’
[underground]
station = IXYZABA5
password = secret

These sections contain information such as passwords and station IDs needed to upload data to weather services. The names of the data entries depend on the service. The example shown is for Weather Underground.

station is the PWS ID allocated to your weather station by Weather Underground.

password is your Weather Underground password.

status.ini - status file format

This file is written by pywws and should not (usually) be edited. The following sections are currently in use:

  • fixed: values copied from the weather station’s “fixed block”.
  • clock: synchronisation information.
  • last update: date and time of most recent task completions.
fixed: values copied from the weather station’s “fixed block”
[fixed]
fixed block = {...}

fixed block is all the data stored in the first 256 bytes of the station’s memory. This includes maximum and minimum values, alarm threshold settings, display units and so on.

clock: synchronisation information
[clock]
station = 1360322930.02
sensor = 1360322743.69

These values record the measured times when the station’s clock logged some data and when the outside sensors transmitted a new set of data. They are used to try and prevent the USB interface crashing if the computer accesses the weather station at the same time as either of these events, a common problem with many EasyWeather compatible stations. The times are measured every 24 hours to allow for drift in the clocks.

last update: date and time of most recent task completions
[last update]
hourly = 2013-05-30 19:04:15
logged = 2013-05-30 19:04:15
daily = 2013-05-30 09:04:15
openweathermap = 2013-05-30 18:59:15
underground = 2013-05-30 18:58:34
metoffice = 2013-05-30 18:59:15
12 hourly = 2013-05-30 09:04:15

These record date & time of the last successful completion of various tasks. They are used to allow unsuccessful tasks (e.g. network failure preventing uploads) to be retried after a few minutes.


Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

Understanding pywws log files

The pywws software uses Python’s logging system to report errors, warnings and information that may be useful in diagnosing problems. When you run the software interactively these messages are sent to your terminal window, when running a daemon process or cron job they should be written to a log file.

This document explains some of the pywws logging messages. It’s not a complete guide, and the messages you see will depend on your weather station and configuration, but it should help you understand some of the more common ones.

Many pywws commands have a -v or --verbose option to increase the verbosity of the logging messages. This option can be repeated for even more verbosity, which may be useful when trying to diagnose a particular fault.

Here are some typical logging outputs. The first shows pywws being run interactively:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
jim@firefly ~ $ pywws-livelog -v ~/weather/data/
08:50:43:pywws.Logger:pywws version 15.07.1.dev1308
08:50:43:pywws.Logger:Python version 2.7.3 (default, Mar 18 2014, 05:13:23) [GCC 4.6.3]
08:50:44:pywws.WeatherStation.CUSBDrive:using pywws.device_libusb1
08:50:49:pywws.Calib:Using user calibration
08:50:49:pywws.Tasks.RegularTasks:Starting asynchronous thread
08:51:05:pywws.ToService(wetterarchivde):1 records sent
08:51:06:pywws.ToService(underground_rf):1 records sent
08:51:06:pywws.ToService(cwop):1 records sent
08:51:52:pywws.ToService(wetterarchivde):1 records sent
08:51:52:pywws.ToService(underground_rf):1 records sent
08:52:40:pywws.ToService(wetterarchivde):1 records sent
08:52:40:pywws.ToService(underground_rf):1 records sent
08:53:28:pywws.ToService(wetterarchivde):1 records sent
08:53:28:pywws.ToService(underground_rf):1 records sent

Note that each line begins with a time stamp, in local time. Line 1 is the command used to start pywws. Line 2 shows the pywws version. Line 3 shows the Python version. Line 4 shows which Python USB library is being used. Line 5 shows that a "user calibration" routine is being used. Line 6 shows that a separate thread is being started to handle uploads (see config: miscellaneous system configuration). The remaining lines show uploads to various weather “services” (see How to integrate pywws with various weather services). You can see from the time stamps that they happen at 48 second intervals.

When running pywws as a daemon process the logging is less verbose:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
2015-07-20 10:46:00:pywws.Logger:pywws version 15.07.1.dev1308
2015-07-20 10:50:06:pywws.weather_station:live_data log extended
2015-07-20 16:25:59:pywws.weather_station:setting station clock 59.637
2015-07-20 16:25:59:pywws.weather_station:station clock drift -0.461377 -0.296364
2015-07-20 16:30:24:pywws.ToService(wetterarchivde):<urlopen error [Errno -2] Name or service not known>
2015-07-20 16:30:24:pywws.ToService(underground_rf):<urlopen error [Errno -2] Name or service not known>
2015-07-20 23:01:16:pywws.ToService(openweathermap):<urlopen error [Errno -2] Name or service not known>
2015-07-21 01:14:18:pywws.weather_station:setting sensor clock 42.6678
2015-07-21 01:14:18:pywws.weather_station:sensor clock drift -2.03116 -1.98475
2015-07-21 09:00:47:pywws.ToTwitter:('Connection aborted.', gaierror(-2, 'Name or service not known'))
2015-07-21 09:00:55:pywws.Upload:[Errno -2] Name or service not known
2015-07-21 09:01:05:pywws.ToService(cwop):[Errno -3] Temporary failure in name resolution
2015-07-21 09:06:05:pywws.ToService(underground):<urlopen error [Errno -2] Name or service not known>
2015-07-21 09:06:05:pywws.ToService(metoffice):<urlopen error [Errno -2] Name or service not known>
2015-07-21 16:30:59:pywws.weather_station:setting station clock 59.4771
2015-07-21 16:30:59:pywws.weather_station:station clock drift -0.159373 -0.262116

Each line begins with a date and time stamp, in local time. Line 1 shows the pywws version. The remaining lines show normal status messages that are described below.

Clock drift
2015-08-31 20:10:45:pywws.weather_station:setting station clock 45.7137
2015-08-31 20:10:45:pywws.weather_station:station clock drift -0.0171086 -0.313699
2015-09-01 01:54:59:pywws.weather_station:setting sensor clock 35.2755
2015-09-01 01:54:59:pywws.weather_station:sensor clock drift -1.12118 -1.37694

These lines report how the weather station’s internal (“station”) and external (“sensor”) clocks are drifting with respect to the computer’s clock. These measurements are used to avoid accessing the station’s USB port at the same time as it is receiving data or logging data, as this is known to cause some station’s USB ports to become inaccessible. The two “drift” figures are the current value (only accurate to about 1 second) and the long term average. You should ensure that the usb activity margin value in your weather.ini file is at least 0.5 seconds greater than the absolute value of the long term drift of each clock. Note that these drift values change with temperature.

The clock drifts are measured at approximately 24 hour intervals. If pywws loses synchronisation with your station it will measure them again. Doing this measurement increases the risk of causing a USB lockup, so if pywws often loses synchronisation you should try and find out why it’s happening.

Network problems

Occasionally one or more of the services and web sites you upload data to may become unavailable. This leads to error messages like these:

2015-08-03 04:19:49:pywws.ToService(underground_rf):[Errno 104] Connection reset by peer
2015-08-03 04:49:27:pywws.ToService(underground_rf):<urlopen error [Errno -2] Name or service not known>
2015-08-03 05:19:41:pywws.ToService(wetterarchivde):<urlopen error [Errno 101] Network is unreachable>
2015-08-03 05:19:46:pywws.ToService(underground_rf):<urlopen error [Errno 101] Network is unreachable>
2015-08-03 05:50:52:pywws.ToService(wetterarchivde):<urlopen error [Errno -2] Name or service not known>
2015-08-03 05:50:52:pywws.ToService(underground_rf):<urlopen error [Errno -2] Name or service not known>

To avoid swamping the log files duplicate messages are not logged, so you cannot tell how long the network outage lasted from the log files.

Status
2015-09-01 21:50:21:pywws.weather_station:status {'unknown': 0, 'invalid_wind_dir': 2048, 'lost_connection': 64, 'rain_overflow': 0}

The raw weather station data includes some “status” bits. If any of these bits is non-zero when pywws starts, or the status changes value when pywws is running, the status value is logged. The most common problem is lost_connection: the weather station console is not receiving data from the outside sensors. Contact is often restored a few minutes later, but if not you may need to reset your weather station console by taking its batteries out. The invalid_wind_dir bit indicates that the wind direction sensor value is missing or invalid. The rain_overflow bit is set when the rain gauge counter has reached its maximum value and gone back to zero.

Please let me know if you ever get a non-zero value for unknown, particularly if you are able to correlate it with some other event. There are 6 bits of data in the status byte whose function is not yet known.

Log extended
2015-08-10 08:25:59:pywws.weather_station:live_data log extended
2015-08-10 08:41:59:pywws.weather_station:live_data log extended
2015-08-10 08:57:59:pywws.weather_station:live_data log extended

This shows a curiosity in the weather station’s internal processing. As the internal and external sensors drift there comes a time when an external reading is expected at the same time as the station is due to log some data. To avoid a clash the station delays logging by one minute. As the external readings are at 48 second intervals this avoids the problem until 16 minutes later (with the normal 5 minute logging interval) when another one minute delay is needed. Eventually the clocks drift apart and normal operation is resumed.

Rain reset
2015-08-25 13:30:51:pywws.Process.HourAcc:2015-08-25 12:30:48 rain reset 1048.4 -> 1047.1
2015-08-25 13:35:51:pywws.Process.HourAcc:2015-08-25 12:30:48 rain reset 1048.4 -> 1047.1
2015-08-25 13:40:51:pywws.Process.HourAcc:2015-08-25 12:30:48 rain reset 1048.4 -> 1047.1

The raw rainfall data from the outside sensors is the total number of times the “see saw” has tipped since the external sensors were last reset (by a battery change, unless you do it quickly). This number should only ever increase, so the pywws.Process module warns of any decrease in the value as it may indicate corrupted data that needs manually correcting. The logging message includes the UTC time stamp of the problem data to help you find it.

Live data missed
1
2
3
4
5
2015-10-30 04:48:19:pywws.ToService(underground_rf):1 records sent
2015-10-30 04:49:07:pywws.ToService(underground_rf):1 records sent
2015-10-30 04:49:56:pywws.weather_station:live_data missed
2015-10-30 04:50:44:pywws.ToService(underground_rf):1 records sent
2015-10-30 04:51:31:pywws.ToService(underground_rf):1 records sent

Line 3 indicate that pywws failed to capture live data.

There are two possible causes. One is that a new data record is identical to the previous one so pywws doesn’t detect a change. This is unlikely to happen if you are receiving wind data properly.

The more likely reason is that processing the previous record took so long that the next one arrived when pywws wasn’t ready for it. “Processing” can include uploading to the Internet which is often prone to delays. A solution to this is to set “asynchronous” to True in weather.ini. This uses a separate thread to do the uploading.

You may run with higher verbosity to get more information. The “pause” values should indicate how soon it’s ready for the next data.

Note that this is just an occasional missing “live” record though, so if it does not happen often you shouldn’t worry too much about it.

“Live log” synchronisation

If you run pywws at a high verbosity you may see messages like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
10:26:05:pywws.Logger:pywws version 15.07.0.dev1307
10:26:05:pywws.Logger:Python version 2.7.8 (default, Sep 30 2014, 15:34:38) [GCC]
10:26:05:pywws.WeatherStation.CUSBDrive:using pywws.device_libusb1
10:26:06:pywws.Calib:Using user calibration
10:26:06:pywws.Tasks.RegularTasks:Starting asynchronous thread
10:26:06:pywws.weather_station:read period 5
10:26:06:pywws.weather_station:delay 2, pause 0.5
10:26:07:pywws.weather_station:delay 2, pause 0.5
10:26:08:pywws.weather_station:delay 2, pause 0.5
10:26:08:pywws.weather_station:delay 2, pause 0.5
10:26:09:pywws.weather_station:delay 2, pause 0.5
10:26:10:pywws.weather_station:delay 2, pause 0.5
10:26:10:pywws.weather_station:delay 2, pause 0.5
10:26:11:pywws.weather_station:delay 2, pause 0.5
10:26:12:pywws.weather_station:delay 2, pause 0.5
10:26:12:pywws.weather_station:delay 2, pause 0.5
10:26:13:pywws.weather_station:delay 2, pause 0.5
10:26:14:pywws.weather_station:delay 2, pause 0.5
10:26:14:pywws.weather_station:live_data new data
10:26:14:pywws.weather_station:setting sensor clock 38.7398
10:26:14:pywws.weather_station:delay 3, pause 45.4993
10:26:16:pywws.Tasks.RegularTasks:Doing asynchronous tasks
10:27:00:pywws.weather_station:delay 3, pause 0.5
10:27:00:pywws.weather_station:avoid 3.83538614245
10:27:04:pywws.weather_station:live_data new data
10:27:04:pywws.weather_station:delay 3, pause 43.3316
10:27:06:pywws.Tasks.RegularTasks:Doing asynchronous tasks
10:27:48:pywws.weather_station:delay 3, pause 0.5
10:27:48:pywws.weather_station:avoid 3.79589626256
10:27:52:pywws.weather_station:live_data new data
10:27:52:pywws.weather_station:delay 4, pause 0.5
10:27:53:pywws.weather_station:delay 4, pause 0.5
10:27:54:pywws.weather_station:delay 4, pause 0.5
10:27:54:pywws.weather_station:delay 4, pause 0.5
10:27:54:pywws.Tasks.RegularTasks:Doing asynchronous tasks
10:27:55:pywws.weather_station:delay 4, pause 0.5
10:27:56:pywws.weather_station:delay 4, pause 0.5
10:27:56:pywws.weather_station:delay 4, pause 0.5
10:27:57:pywws.weather_station:delay 4, pause 0.5
10:27:58:pywws.weather_station:delay 4, pause 0.5
10:27:58:pywws.weather_station:delay 4, pause 0.5
10:27:59:pywws.weather_station:delay 4, pause 0.5
10:28:00:pywws.weather_station:delay 4, pause 0.5
10:28:00:pywws.weather_station:delay 4, pause 0.5
10:28:01:pywws.weather_station:delay 4, pause 0.5
10:28:02:pywws.weather_station:delay 4, pause 0.5
10:28:02:pywws.weather_station:delay 4, pause 0.5
10:28:03:pywws.weather_station:delay 4, pause 0.5
10:28:04:pywws.weather_station:delay 4, pause 0.5
10:28:04:pywws.weather_station:delay 4, pause 0.5
10:28:05:pywws.weather_station:delay 4, pause 0.5
10:28:06:pywws.weather_station:delay 4, pause 0.5
10:28:06:pywws.weather_station:delay 4, pause 0.5
10:28:07:pywws.weather_station:live_data new ptr: 007320
10:28:07:pywws.weather_station:setting station clock 7.43395
10:28:07:pywws.weather_station:avoid 1.91954708099
10:28:10:pywws.DataLogger:1 catchup records
10:28:10:pywws.Process:Generating summary data
10:28:10:pywws.Process:daily: 2015-08-31 21:00:00
10:28:10:pywws.Process:monthly: 2015-07-31 21:00:00
10:28:10:pywws.Process:monthly: 2015-08-31 21:00:00
10:28:10:pywws.weather_station:delay 0, pause 26.121
10:28:12:pywws.Tasks.RegularTasks:Doing asynchronous tasks

Line 6 shows that the weather station has the usual 5 minute logging interval. Lines 7 to 18 show pywws waiting for the station to receive data from the outside sensors. The delay value is the number of minutes since the station last logged some data. The pause value is how many seconds pywws will wait before fetching data from the station again. Lines 19 & 20 show new data being received and the “sensor” clock being set. Line 21 shows that pywws now knows when data is next expected, so it can sleep for 43 seconds. Line 22 shows the separate “upload” thread doing its processing while the main thread is sleeping. Line 24 shows pywws avoiding USB activity around the time the station should receive external data. Lines 31 to 53 show pywws waiting for the station to log data. Lines 54 & 55 show the station logging some data and pywws using this to set the “station” clock. The 6 digit number at the end of line 54 is the hexadecimal address where “live” data will now be written to, leaving data at the previous address as a “logged” value. Lines 57 to 61 show pywws fetching logged data from the station and then processing it to produce the various summaries.

Crash with traceback

Sometimes pywws software crashes. When it does, the log file will often contain a traceback like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
18:50:57:pywws.LiveLog:error sending control message: Device or resource busy
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pywws/LiveLog.py", line 80, in LiveLog
    logged_only=(not tasks.has_live_tasks())):
  File "/usr/local/lib/python2.7/dist-packages/pywws/LogData.py", line 256, in live_data
    for data, ptr, logged in self.ws.live_data(logged_only=logged_only):
  File "/usr/local/lib/python2.7/dist-packages/pywws/WeatherStation.py", line 446, in live_data
    new_ptr = self.current_pos()
  File "/usr/local/lib/python2.7/dist-packages/pywws/WeatherStation.py", line 585, in current_pos
    self._read_fixed_block(0x0020), self.lo_fix_format['current_pos'])
  File "/usr/local/lib/python2.7/dist-packages/pywws/WeatherStation.py", line 641, in _read_fixed_block
    result += self._read_block(mempos)
  File "/usr/local/lib/python2.7/dist-packages/pywws/WeatherStation.py", line 629, in _read_block
    new_block = self.cusb.read_block(ptr)
  File "/usr/local/lib/python2.7/dist-packages/pywws/WeatherStation.py", line 265, in read_block
    if not self.dev.write_data(buf):
  File "/usr/local/lib/python2.7/dist-packages/pywws/device_pyusb.py", line 152, in write_data
    usb.REQ_SET_CONFIGURATION, buf, value=0x200, timeout=50)
USBError: error sending control message: Device or resource busy

Line 1 shows the exception that caused the crash. Lines 3 to 18 show where in the program the problem happened. Usually the last one is of interest, but the other function calls show how we got there. Line 19 shows the full exception. In this case it’s a USBError raised by the pyusb library.


Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

Humidity Index (Humidex)

Auteur van deze sectie: Rodney Persky

Background

Using your weather station can be fun, and reporting daily to various weather data sites can be very useful for your neighbours to check out the weather. However, at some point you may want to know how the weather effects your body, and if there is a way to tell when it’s good or not to work outdoors.

Here enters a whole realm of calculations based on energy transferring though walls, and the resistance offered by them. It can be a great learning adventure, and can save you a great deal of money, finding out how energy moves around.

Introduction

Humidex is a tool to determine how an individuals body will react to the combination of Wind, Humidity and Temperature. The background of which is a heat balance across from your midriff to your skin, and is complimentary to ISO 7243 “Hot Environments - Estimation of the heat stress on working man”. A few important notes,

  • These indices are based off a number of assumptions which may result in over or under-estimation of your bodies internal state
  • A personal weather station may not show the correct conditions, and may have an over or under estimation of the humidity, wind or temperature
  • Clothing choices effect the personal fatigue and the bodies ability to reject heat, a low Humidity Index doesn’t mean you can wear anything
  • An individuals fitness will effect their bodies response to changing temperature, and experience will aid in knowing when to stop working
  • The duration of activities that can be performed requires knowledge on the intensity, which cannot be represented though this index
Assumptions

There are a number of assumptions that have been made to make this work which will directly affect its useability. These assumptions however have not been made available from Environment Canada, who are the original developers of the Humidex used in the PYWWS function cadhumidex. It is safe enough however to say that the following would have been some assumptions:

  • Clothing type, thickness
  • Skin area exposed to free air
  • Sun exposure

However, there are a number of assumptions pywws needs to make in calculating the Humidex:

  • The humidity, wind and temperature readings are correct

There are also assumptions about the individuals body type and ‘acclimatisation’

  • An individuals fitness will effect their bodies response to changing temperature
  • Experience will aid in knowing when to stop working
Important References

Being Prepared for Summer - http://www.ec.gc.ca/meteo-weather/default.asp?lang=En&n=86C0425B-1

How to use

The function is descriptively named cadhumidex and has the parameters temperature and humidity, essentially the function operates as a conversion and can be used in a straightforward manner:

<ycalc>cadhumidex(data['temp_out'],data['hum_out'])</ycalc>

Putting it together, I have added colours that follow basic warning colors and the different brackets to produce a decent graph:

<?xml version="1.0" encoding="ISO-8859-1"?>
<graph>
  <title>Humidity Index, Bands indicate apparent discomfort in standard on-site working conditions</title>
  <size>1820, 1024</size>
  <duration>hours=48</duration>
  <xtics>2</xtics>
  <xformat>%H%M</xformat>
  <dateformat></dateformat>
  <plot>
    <yrange>29, 55</yrange>
    <y2range>29, 55</y2range>
    <ylabel></ylabel>
    <y2label>Humidex</y2label>
    <source>raw</source>
    <subplot>
      <title>Humidex</title>
      <ycalc>cadhumidex(data['temp_out'],data['hum_out'])</ycalc>
      <colour>4</colour>
      <axes>x1y2</axes>
    </subplot>
    <subplot>
      <title>HI > 54, Heat Stroke Probable</title>
      <ycalc>54</ycalc>
      <axes>x1y2</axes>
      <colour>1</colour>
    </subplot>
    <subplot>
      <title>HI > 45, Dangerous</title>
      <ycalc>45</ycalc>
      <axes>x1y2</axes>
      <colour>8</colour>
    </subplot>
    <subplot>
      <title>HI > 40, Intense</title>
      <ycalc>40</ycalc>
      <axes>x1y2</axes>
      <colour>6</colour>
    </subplot>
    <subplot>
      <title>HI > 35, Evident</title>
      <ycalc>35</ycalc>
      <axes>x1y2</axes>
      <colour>2</colour>
    </subplot>
    <subplot>
      <title>HI > 30, Noticeable</title>
      <ycalc>30</ycalc>
      <axes>x1y2</axes>
      <colour>3</colour>
    </subplot>
  </plot>
</graph>
Not running the latest update?

If you are not running the latest update / do not want to, then this can be implemented using a longer <ycalc> as follows:

<ycalc>data['temp_out']+0.555*(6.112*10**(7.5*data['temp_out']/(237.7+data['temp_out']))*data['hum_out']/100-10)</ycalc>

Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.


Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

Python programma´s en modules

Instellen en configureren pywws
pywws.TestWeatherStation Test verbinding met weerstation.
pywws.SetWeatherStation Stel enkele weerstation parameters in.
pywws.version Toon pywss versieinformatie.
pywws.Reprocess
pywws.TwitterAuth Sta pywws toe berichten te plaatsen op je Twitter account.
pywws.USBQualityTest Test kwaliteit van USB verbinding naar weerstation
pywws.EWtoPy
Verkrijg gegevens en verwerk ze
pywws.Hourly
pywws.LiveLog
pywws.livelogdaemon
‘ Interne’ modules
pywws.Tasks
pywws.LogData Sla weerstation geschiedenis op naar bestand.
pywws.Process
pywws.calib Calibreer onbewerkte weerstation gegevens.
pywws.Plot
pywws.WindRose
pywws.Template
pywws.Forecast
pywws.ZambrettiCore
pywws.Upload Upload files to a web server by ftp or copy them to a local directory
pywws.ToTwitter Post a message to Twitter
pywws.toservice
pywws.YoWindow
pywws.WeatherStation Get data from WH1080/WH3080 compatible weather stations.
pywws.device_libusb1 Low level USB interface to weather station, using python-libusb1.
pywws.device_pyusb1 Low level USB interface to weather station, using PyUSB v1.0.
pywws.device_pyusb Low level USB interface to weather station, using PyUSB v0.4.
pywws.device_ctypes_hidapi Low level USB interface to weather station, using ctypes to access hidapi.
pywws.device_cython_hidapi Low level USB interface to weather station, using cython-hidapi.
pywws.DataStore DataStore.py - stores readings in easy to access files
pywws.TimeZone
pywws.Localisation Localisation.py - provide translations of strings into local
pywws.conversions
pywws.Logger Common code for logging info and errors.
pywws.constants Bits of data used in several places.

Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.

Indices and tables

Credits

I would not have been able to get any information from the weather station without access to the source of Michael Pendec’s “wwsr” program. I am also indebted to Dave Wells for decoding the weather station’s “fixed block” data.

Last of all, a big thank you to all the pywws users who have helped with questions and suggestions, and especially to those who have translated pywws and its documentation into other languages.

Legalese

pywws - Python software for USB Wireless WeatherStations.
Copyright (C) 2008-15 pywws contributors

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA


Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.