PACT Python Bindings

Lee Taylor
Greg Smethells
Al Franz


Introduction

PDB

PGS


Introduction

PyPact provides access to PACT from Python. This is provided as a package with two modules. The package is pact and the modules are pdb and pgs. The pdb module provides access to Score, PML and PDBLib. The pgs module provides access to Score, PML, PDBLib and PGS. Only one module should be used at a time. No access is provided to Scheme, PPC or Panacea at this time.

TODO
If you are using a PACT distribution:
(The user needs to know how to build the shared objects.)
If you see the error message:

ImportError: No module named _pdb

then you need to build the shared objects.
END OF TODO

If you are using your own private version of PACT built from a PACT distribution, e.g. in ~/pact, then you can do the following in your Python code:

Example

import os
import sys
site_packages = os.path.expanduser( "~/pact/python" )
if os.path.isdir( site_packages ) :
    sys.path.append( site_packages )
else :
    print "No PACT python support on this platform."
    sys.exit( -1 )
                                                                                
import pact.pdb as pdb

Putting the modules in a package helps keep the namespace clean and avoids confusion with the other pdb module - the Python Debugger.

To import the library:

import pact.pdb as pdb
or
import pact.pgs as pgs
To import the PDB library AND use the Python debugger:
import pact.pdb
import pdb

Recall that the PACT tools fit in the following hierarchy:

ULTRA SX
PANACEA
PGS SCHEME PPC
PDB
PML
SCORE

Score and PML are not provided as stand alone modules since PDB is the first level that Python users are usually interested in using. Much of Score functionality such as memory allocation, hash tables and association lists are accessable from Python. However the module also depends on PDBLib's functionality to store arbitrary data in Score's hash tables and association lists. In the C library this is implemented by associating a type with each haelem or pcons. In PyPact the type must be defined in a PDB file. PyPact uses a virtual internal file to store the type information. It is accessed as the module variable vif. It is also the default file argument for many methods.

Many structures in PACT are represented as classes in PyPact. Most structures have a function in the C API that returns a pointer to a new structure and several support functions that receive the structure pointer as one of the arguments (usually the first). This maps directly to a constructor and methods.

The hash table functions follow the typical pattern.

C Bindings:

hasharr *SC_hash_alloc()
int SC_hasharr_install(hasharr *self, char *type, void *key, void *obj)
void SC_free_hasharr(hasharr *self)
With typical usage:
     int ierr, ival1;
     char one[] = "one";
     hasharr *ht;

     ht = SC_hash_alloc();
  
     ival1 = 1;
     iok = SC_hasharr_install(ht, "int", "one", &ival1);
     if (iok == 0) {
        errproc("Error inserting one\n";); 
     }

     SC_free_hasharr(ht);
In PyPact SC_make_hasharr is replaced by the hasharr class constructor and SC_hasharr_install becomes a method of the instance. SC_hash_free is called by Python's garbage collector when there are no more references to the instance.
    import pact.pdb as pdb
    ht = pdb.hasharr()
    ht.install("one", 1)
PACT errors are trapped by raising Python exceptions.

Where natural, types also use standard protocols. For example, both hash tables and association lists use the mapping protocal.

    ht = pdb.hasharr()
    ht["one"] = 1

PDB

This section focuses on how to use the PDB module.

The pdbdata Object

The pdbdata object is the only object in PyPact that does not map directly to a structure in PACT. It is used to fully describe a piece of memory and acts as a middle-man between the C and Python type systems. It consists of a pointer to memory and sufficient information to describe the memory. This includes the type and dimensions. This also includes a pointer to a PDBfile that is used to look up the type. With this information Python is able to access the memory as the user expects including accessing members of a derived type as attributes of an object. Python provides int and float objects. The Python implementation uses a C long for int and and C double for float. The pdbdata constructor allows the default types to be overridden. pdbdata(1, 'int') Since Python does not support pointers directly, a sequence of Python objects will match several C declarations. [ 1, 2, 3, 4, 5] can be both int[5] and int *. In one case, it is clear that there are 5 integers (int[5]). In the other, we only know that it points to an int. By default, a pdbdata object will be explicit about lengths and use int[5]. However, if it's necessary to match a specific type, then int * can be specified. The pdbdata type requires that pointers be allocated by Pact. This allows it to find out how much memory is allocated which allows a pdbdata to be converted back into Python objects. Nested sequences are represented as multiply dimensioned types. ([[1,2,3],[4,5,6]] is long[2][3]. If the array is not regular, then it is treated as an array of pointers. [[1,2,3],[4,5]] is long *[2]. It is also possible to convert Python class instances into pdbdata object. See the register_class method of PDBfile object below. None is treated as NULL and results in a pointer type. No type at all defaults to int [None] is int *[1]

Constructor

pdbdata(type, data, file)
data -- data of type type to be encapsulated.
type -- string name of the data type of the pdbdata.
file -- define the types to this PDBfile. (This argument is optional and the virtual internal file is used by default.)

Module Methods

These module methods take a pdbdata object and return the individual fields.

getdefstr(obj) - returns a defstr object.
gettype(obj) - returns a string.
getfile(obj) - returns a PDBfile object.
getdata(obj) - returns a CObject object.
In addition to using the constructor, a pdbdata can be returned by any method that returns user data; for example, pdbfile.read.

It is controlled by the setform method.

setform(array, struct, scalar)
With no arguments the current values of setform are returned as a tuple. array, struct and scalar should be one of the following values, and reflect how reads and writes handle arrays, structs, and scalar data, respectively.
AS_NONE - Keep existing value.
AS_PDBDATA - return as a pdbdata
AS_OBJECT - return as object
AS_TUPLE - return as a tuple
AS_LIST - return as a list
AS_DICT - return as a dictionary
AS_ARRAY - return as a numpy array
Not all constants are valid for all forms.
type array struct scalar
AS_PDBDATA X X X
AS_OBJECT X
AS_TUPLE X X
AS_LIST X
AS_DICT X
AS_ARRAY X

unpack(data, array, struct, scalar)
Unpack a pdbdata object into more Python friendly types.

If pdbdata represents a structure then the members can be accessed as attributes of object.

If pdbdata represents an array then the sequence protocol is supported.

Examples

>>> import pact.pdb as pdb

>>> d = pdb.pdbdata(4.0, 'double')
>>> d
data =   4.0000000000000000e+00

>>> r = pdb.unpack(d)
>>> type(r)
<type 'float'>
>>> r
4.0

>>> d = pdb.pdbdata((4.0, 5.0), 'double[2]')
>>> d
data(0) =   4.0000000000000000e+00
data(1) =   5.0000000000000000e+00

>>> r = pdb.unpack(d)
>>> type(r)
<type 'list'>
>>> r
[4.0, 5.0]
>>> pdb.setform(array=pdb.AS_TUPLE)
(3, 3, 2)
>>> r = pdb.unpack(d)
>>> type(r)
<type 'tuple'>
>>> r
(4.0, 5.0)


>>> d = pdb.pdbdata([None, [4., 5.]], 'double **')
>>> d
data(0) = (nil)
data(1)(0) =   4.0000000000000000e+00
data(1)(1) =   5.0000000000000000e+00

>>> r = pdb.unpack(d)
>>> r
[None, [4.0, 5.0]]
>>> print r[0]
None
>>> print r[1]
[4.0, 5.0]

File Object

There are a few module variables that are used to access files. files is a dictionary of opened PDBfile instances indexed by file name. It is used to store references to files to avoid garbage collection until the close method is explicitly called. vif is the virtual internal file. It is used as the default file for many other methods.

PDBfile(name[, mode]) name is the name of the file to open/create. mode is the file mode - r read, w write, a append The default is r. open is an alias for PDBfile.

Attributes

object
name
type
symtab
chart
host_chart
attrtab
previous_file
date
mode
default_offset
virtual_internal
system_version
major_order

Methods

The ind argument is used to index arrays. If it is a scalar, it must be the number of items. If it is a sequence, then each member applies to a dimension. If a scalar, then it is the number of items. If length is 1, it is interpreted as (upper, ). If length is 2, it is interpreted as (lower, upper). If length is 3, it is interpreted as (lower, upper, stride).

The following examples use f90 array notation.
f90 ind
(4) 4
(4) [4]
(0:5) [(0,5)]
(1:5:2) [(1,5,2)]
(4, 5) [4, 5]
(0:5, 0:6) [(0,5), (0,6)]
(7, 0:10) [7, (0, 10)]
When writing data, PyPact will try to determine the type of the data if outtype is not specified. Python reals map to doubles. strings map to char *

flush()
close()
write(name, var[, outtype, ind])
name is the name of the variable to write.
var is the variable object to write.
outtype optional, is the type of variable.
ind optional, indexing information.
write_raw(name, var, type[, ind])
name is the name of the variable to write.
var is the variable object to write. It must support the buffer interface.
type is the type of variable.
ind optional, indexing information.
read(name[, intype, ind])
name is the name of the variable to read.
intype optional, type of data desired.
ind optional, indexing information.
defent(name, type)
name name of variable.
type type of data in file.
defstr(name, members)
name name of new type
members optional, sequence of types. If members are defined a new type is created. Otherwise name is looked up and returned.
cd(dirname)
dirname Name of directory.
mkdir(dirname)
dirname Name of directory.
ln(var, link)
var
link
ls([path, type])
path optional
type optional
pwd()
register(cls, type [, ctor])
cls The Python Class object
type The name of the pdblib defstr.
ctor This function is used during read. It accepts a dictionary of structure members and returns an object of class cls.

Examples

Open a file.

>>> import pact.pdb as pdb
>>> fp = pdb.open("xxxx", "w")
>>> type(fp)
<type 'PDBfile'>
>>> pdb.files
{'xxxx': <PDBfile object at 0xb3f7bb40>}
>>> fp.close()
>>> pdb.files
{}
>>>  ctrl-D
Write four doubles to a files as a 2-d double array and as a 1-d float array
%python
>>> import pact.pdb as pdb
>>> fp = pdb.open("xxxx", "w")
>>> ref = [2.0, 3.0, 4.0, 5.0]
>>> fp.write("d2", ref, ind=[2,2])
>>> fp.write("d3", ref, "float")
>>> fp.close()
>>>  ctrl-D
%pdbview xxxx

	PDBView 2.0 - 11.22.04

-> ls

d2   d3                                                                         
-> d2

/d2(0,0) =   2.0000000000000000e+00
/d2(0,1) =   3.0000000000000000e+00
/d2(1,0) =   4.0000000000000000e+00
/d2(1,1) =   5.0000000000000000e+00

-> d3

/d3(0) =   2.0000000e+00
/d3(1) =   3.0000000e+00
/d3(2) =   4.0000000e+00
/d3(3) =   5.0000000e+00

-> desc d2

Name: d2
Type: double
Dimensions: (0:1, 0:1)
Length: 4
Address: 108

-> desc d3

Name: d3
Type: float
Dimensions: (0:3)
Length: 4
Address: 140

-> quit
Write a class instance
%cat user.py
class User:
    def __init__(self, a, b, c):
        self.a = a
        self.b = b
        self.c = c
    def __repr__(self):
        return 'User(%(a)d, %(b)d, %(c)f)' % self.__dict__

def makeUser(dict):
    return User(dict['a'], dict['b'], dict['c'])

fp = pdb.open("user.pdb", "w")
fp.defstr( "user", ("int a", "int b", "float c"))
fp.register_class(User, "user", makeUser)

v1 = User(1,2,3)
fp.write("var1", v1)
v2 = fp.read("var1")
fp.close()
print "v1 =", v1
print "v2 =", v2

%python user.py
v1 = User(1, 2, 3.000000)
v2 = User(1, 2, 3.000000)

%pdbview user.pdb
 
-> var1
 
/var1.a =          1
/var1.b =          2
/var1.c =  3.0000000e+00


Defstr Object

A defstr type is used to define and create structures. defstr(name, members[, file])
name name for defstr.
members sequence of type names.
file PDBfile instance. optional defaults to vif
The returned instance is also a constructor for instances of a defstr.

Attributes

dp
type
size_bits
size
alignment
n_indirects
is_indirect
convert
onescmp
unsgned
order_flag

Methods

A defstr supports the mapping protocol plus some methods usually associated with mappings.
has_key Not Implemented
items Not Implemented
keys
values Not Implemented
get Not Implemented

Examples


>>> import pact.pdb as pdb
>>> d = pdb.defstr('struct', ('int i', 'float j'))
>>> type(d)

>>> print d
Type: struct
Alignment: 4
Members: {int i;
          float j;}
Size in bytes: 8

>>> a = d((3,4))
>>> a
data.i =           3
data.j =   4.0000000e+00

>>> a.i
3
>>> a.j
4.0
>>> a.i = 5
>>> a.j = 6
>>> a
data.i =           5
data.j =   6.0000000e+00

-----------------

>>> import pact.pdb as pdb
>>> d = pdb.defstr('struct', ('int i', 'float j'))
>>> a = d((3,4))
>>> fp = pdb.open('yyyy', 'w')
>>> fp.defstr('struct', d)
Type: struct
Alignment: 4
Members: {int i;
          float j;}
Size in bytes: 8

>>> fp.write('aaa', a)
>>> fp.close()
>>>  ctrl-D
% pdbview yyyy

	PDBView 2.0 - 11.22.04

-> ls

aaa                                                                             
-> aaa

/aaa(1).i =           3
/aaa(1).j =   4.0000000e+00

-> desc aaa

Name: aaa
Type: struct
Dimensions: (1:1)
Length: 1
Address: 108

-> struct struct

Type: struct
Alignment: 4
Members: {int i;
          float j;}
Size in bytes: 8

Memdes Object

Attributes

desc
member
cast_memb
cast_offs
is_indirect
type
base_type
name
number

Methods

None

Examples




SCORE

hasharr and assoc both seem very similar at the Python level. Both implement the mapping protocol. The chief difference is in the data structures they build in memory. This allows the user to build the type of data structure required by PACT. For example, many graphics routines take or return an association list to describe plotting options. The application can treat the association list in a Pythonic manner by treating it as a dictionary.

Memory Allocation

PyPact provides a way to allocate and check memory using PACT's memory allocation routines. The void * pointer is contained in a CObject. There is no PyPact specific type/class for memory. All methods are module methods, not class methods. The CObject is a convient way to pass C pointers around and is generally only useful to methods that know what to do with it.

Attributes

None

Methods

zero_space(flag)
alloc(nitems, bytepitem, name)
realloc(p, nitems, bytepitem)
sfree(p)
mem_print(p)
mem_trace()
reg_mem(p, length, name)
dereg_mem(p)
mem_lookup(p)
mem_monitor(old, lev, id)
mem_chk(type)
is_score_ptr(p)
arrlen(p)
mark(p, n)
ref_count(p)
set_count(p, n)
permanent(p)
arrtype(p, type)

Examples

>>> p = pdb.alloc(2, 8, "array1")
>>> type(p)
<type 'PyCObject'>
>>> pdb.arrlen(p)
16

Hash Table

Attributes

None

Methods

install(key, obj, type)
def_lookup(key)
clear()
has_key(key)
items - not implemented
keys()
update(dict)
values - not implemented
get - not implemented

Examples

>>> ht = pdb.hasharr()
>>> ht["one"] = 1
>>> ht.keys()
('one',)
>>> ht["one"]
1

>>> pdb.vif.chart.keys()
('defstr', 'syment', 'symindir', 'symblock', 'memdes', 'dimdes',
'hasharr', 'haelem', 'PM_mapping', 'PM_mesh_topology', 'PM_set',
'PG_image', 'pcons', 'SC_array', 'Directory', 'function',
'REAL', 'double', 'float', 'u_long_long', 'long_long', 'u_long',
'long', 'u_integer', 'integer', 'u_int', 'int', 'u_short', 'short',
'u_char', 'char', '*')
>>> pdb.vif.symtab.keys()
('/', '/&ptrs/')

Association List

Attributes

None

Methods

clear - not implemented
has_key(key)
items()
keys()
update(dict)
values - not implemented
get - not implemented

Examples




PML

TODO

Mapping Object

Attributes

Methods

Examples


Set Object

Attributes

Methods

Examples


Field Object

Attributes

Methods

Examples


Mesh Topology Object

Attributes

Methods

Examples






PGS

API

PyPact has some user callable routines to allow the developer to define their own types to PyPact. This will allow PyPact to return an instance of the correct class when reading from a file.

typedef int (*PP_pack_func) (void *vr, PyObject *obj, long nitems, int tc)
typedef PyObject *(*PP_unpack_func) (void *vr, long nitems)
typedef PP_descr *(*PP_get_descr)(PP_file *fileinfo, PyObject *obj)

PP_descr *PP_make_descr(
    PP_types      typecode,
    char         *type,
    long          bpi
    )
PP_type_map *PP_make_type_entry(
    PP_types       typecode,
    int            sequence,
    PP_descr      *descr,
    PyTypeObject  *ob_type,
    PP_pack_func   pack,
    PP_unpack_func unpack,
    PP_get_descr   get_descr
    )
void PP_register_type(PP_file *fileinfo, PP_type_entry *entry)
void PP_register_object(PP_file *fileinfo, PP_type_entry *entry)

Developer Notes

This section focuses on areas that developers using PyPact and developers of PyPact might be interested in.

Generating Source

Much of the source code for PyPact is generated using the modulator tool from Basis. This tool generates the boiler plate that the Python API requires to connect functions and data structures together into an extension module. This tool generates new style classes. The input consists of an IDL file (interface definition file). It also reads any previously generated code to preserve changes made to certain sections of the generated source. These sections are contained between DO-NOT-DELETE splicer.begin and DO-NOT-DELETE splicer.end comments.

Each generated file starts with the comment This is generated code.

Installation

PyPact will be installed by dsys if shared libraries are defined. PACT's autoconf/automake system will also build the modules. Finally, a setup.py script is provided to build the module.

The default method of building the extension will load the PACT libraries into the extension. This allows things to work as expected when importing PyPact from the Python executable. If, instead, Python is imbedded into an application which already has the PACT libraries loaded, then importing PyPact will result in two copies of PACT being in memory. This is not a Good Thing. In this case, the application developers will have to rebuild PyPact without loading the PACT libraries. One way to accomplish this is by editing the setup.py script to remove the libraries argument from the Extension constructor.

User Defined Classes

TODO


For questions and comments, please contact the PACT Development Team.
Last Updated: 03/03/2007
llnl logo  
  LLNL-CODE-422942| Privacy & Legal Notice
Last modified: January 28, 2010   
  
Contact: wci-webteam@llnl.gov