Data‎ > ‎

The BDBP

What is the BDBP?

Officially: The Binary Data Base of Profiles
In reality: Birgit's Database of Biblical Proportions

The BDBP is a collection of measurements of atmospheric composition from various satellite-, balloon-, and ground-based measurements.  The data base is described in detail in:
  • Hassler, B., G. E. Bodeker, and M. Dameris (2008), Technical Note: A new global database of trace gases and aerosols from multiple sources of high vertical resolution measurements, Atmospheric Chemistry and Physics, 8, 5403-5421. 
  • Hassler, B., G. E. Bodeker, I. Cionni, and M. Dameris (2009), A vertically resolved, monthly mean, ozone database from 1979 to 2100 for constraining global climate model simulations, International Journal of Remote Sensing, 30(15-16), 4009–4018.

Which version of the BDBP is available?

The current version of the BDBP is version 1.1.0.6. This version is very similar to that described in Hassler et al. (2009) but now also includes the ILAS and ILAS II data. We are currently working on a new version of the database. This will include the AURA MLS data as well as two new grids: one where the vertical coordinate is the altitude above/below the local tropopause, and one where the vertical coordinate if the pressure above/below the local tropopause. If you would like to be made aware of updates to this data base, please email me at greg@bodekerscientific.com and and I will add your name to the list of BDBP users.


How do I get the data that is in the BDBP?

The database comprises 3 sets of 90 files each. If you’ve read the paper you will know why there are 3 sets of 90 files. If you haven’t read the paper – go and read the paper. These files are available here. To access the data you will need a username and password. These can be obtained by emailing datasets@bodekerscientific.com. At present Bodeker Scientific has no financial support to maintain this database and so if there is anyway that you can contribute towards the maintenance of this database, that would be much appreciated. That said, this database is made freely available to any not-for-profit organisation or individual. In return for providing you with this database, all we ask is that you let us know what you plan to do with the database and, if the database is used in a publication, to discussion acknowledgement/co-author options with us.

On the FTP server for this database you will see three different directories, viz.:

EqlatTheta:    These are the data files for the data gridded by equivalent latitude and potential temperature.

LatAlt:                These are the data files for the data gridded by latitude and altitude.

LatPress:          These are the data files for the data gridded by latitude and pressure.

If you have any questions regarding the database, please email me at greg@bodekerscientific.com.


Additionally, Version 1.0 has been archived by NOAA (National Climatic Data Center), and is available here, or via the following DOI: http://ezid.cdlib.org/id/doi:10.7289/V56M34RT
 

How do I add my measurements to the BDBP?

Data can easily be added to the BDBP. To maintain the high vertical resolution of the data base we only accept data at high vertical resolution i.e.~ 1 km or better. We also require that each measurement be provided with a measurement error in percent. If you want to have your data added to the BDBP, please contact us. In general, we will expect a text file listing (in columns separated by spaces):

latitude, longitude, year, month, date, hour, minute, second, altitude, pressure, temperature, measurement ID (see Table 1), value, percentage error

The availability of altitude, pressure, and temperature data will determine which of the three data grids the measurements can be added to.


What code is available for extracting the data from the BDBP?

Reading the binary data files

There are 3 instances of the BDBP, viz.:

I. Where the data are gridded as a function of geographic latitude, longitude, altitude (in 1 km steps) and time,

II. Where the data are gridded as a function of geographic latitude, longitude, pressure (at levels ~1 km apart) and time,

III. Where the data are gridded as a function of equivalent latitude, potential temperature (8 levels from 300K to 650K) and time.

Each instance of the data base comprises 90 files, each spanning 2o of latitude in the cases of Grids I and II and 2o of equivalent latitude in the case of Grid III.

To extract the data from each file repeat the following two step sequence until you get to the end of the file.

  1. Read in a 2 byte unsigned integer (0..65535; type word in Pascal). This tells you what level you are at. For Grid I it is the altitude in km. For Grid II it is the pressure where the pressure is given by 1013.25×exp(-i/7)  where i is the value read in. For Grid III it is the isentropic level where the index runs from 1 to 8 denoting 300K, 315K, 330K, 350K, 400K, 450K, 550K and 650K respectively.
  2. Read in another 2 byte unsigned integer. This tells you how many measurement sets you will now read in.

Reading in a measurement set

To read in a measurement set from a file (this applies to all files and all 3 grids):

  1. Read in a 1 byte unsigned integer (0..255; type byte in Pascal). This will tell you how many characters are in the identifier string. Call this N.
  2. Read in 10 characters, and concatenate the first N into a string. This string will tell you the source of the set of measurements e.g. 'SAGE1'.
  3. Read in an 8 byte double precision floating point value (type double in Pascal). This will tell you the date and time that the set of measurements was made. The integral part of the value is the number of days that have passed since 30 December 1899. The fractional part of the value is the fraction of a 24 hour day since midnight at the start of the day. For example: 0 corresponds to 30 December 1899 12:00 am, 35065.25 corresponds to 1 January 1996 6:00 am.
  4. Read in a 4 byte single precision floating point value (type single in Pascal). This is the latitude at which the measurement set was made.
  5. Read in a 4 byte single precision floating point value. This is the longitude at which the measurement set was made.
  6. Read in a 4 byte single precision floating point value. This is the equivalent latitude at which the measurement set was made. If an equivalent latitude value could not be calculated for the set of measurements, this value will be -1E10.
  7. Read in a 2 byte unsigned integer. This tells you how many measurements in the current measurement set you will now read in.

Reading in a measurement

To read in a measurement from a file (this applies to all files and all 3 grids):

  1. Read in a 4 byte single precision floating point value. This is the value of the measurement. Look at Table 1 to see what the units are.
  2. Read in a 2 byte unsigned integer. This is the percentage error on the measurement multiplied by 100.
  3. Read in a 1 byte unsigned integer. This tells you what type of measurement it is, i.e. the measurement ID (see Table 1).

Keep going. Stop when you get to the end of the file.



Table 1

 ID                  Measurement Description  ID          Measurement Description
 0 The altitude in km  18 The aerosol extinction at 601.4 nm in km-1
 1 The temperature in Kelvin  19 The aerosol extinction at 779.4 nm in km-1 
 2 The pressure in hPa  20 The aerosol extinction at 781 nm in km-1
 3 The O3 concentration in 1018 molecules/m3 (low resolution data)  21 The aerosol extinction at 921 nm in km-1
 4 The NO2 concentration in 1018 molecules/m3  22 The aerosol extinction at 922.4 nm in km-1 
 5 The H2O mixing ratio in 10-6 moles/mole  23 The aerosol extinction at 1018 nm in km-1 
 6 The NO mixing ratio in 10-9 moles/mole  24 The aerosol extinction at 1020 nm in km-1 
 7 The CH4 mixing ratio in 10-6 moles/mole  25 The aerosol extinction at 1060 nm in km-1 
 8 The HCl mixing ratio in 10-9 moles/mole  26 The aerosol surface area density in μm2/cm3 
 9 The HF mixing ratio in 10-9 moles/mole  27 The aerosol effective radius in μm 
 10 The aerosol extinction at 352.3 nm in km-1     28  In the case of a solar occultation measurement whether it was at sunrise (1)  or sunset (2)
 11 The aerosol extinction at 353.4 nm in km-1  29 The relative humidity in percent 
 12 The aerosol extinction at 385 nm in km-1  30 In the case of an ozonesonde based measurement this is the sonde normalization factor. Note that this value has not been applied.
 13 The aerosol extinction at 441.6 nm in km-1  31 The aerosol extinction at 1000 nm in km-1 
 14 The aerosol extinction at 442.3 nm in km-1  32 The aerosol extinction at 450 nm in km-1 
 15 The aerosol extinction at 453 nm in km-1  33 The O3 concentration in 1018 molecules/m3 (high resolution data- available only for ozonesondes) 
 16 The aerosol extinction at 525 nm in km-1     34 The aerosol extinction at 386 nm in km-1 
 17 The aerosol extinction at 603.4 nm in km-1  35 The aerosol extinction at 452 nm in km-1


Here is some Delphi code that implements the recipe outlined above

var

  Source:file;

  NumMeasurementSets,

  NumMeas,

  AltIndex,

  Error:word;

  Test,

  MeasType:byte;

  Ch:char;

  I,J,K:integer;

  DateTime:double;

  Value,

  Latitude,

  Longitude,

  EquivLat:single;

  SourceStr:string;

begin

  assignfile(Source,'D:\archive\BDBP_V1\LatAlt\TimeSeries_21.bin');

  reset(Source,1);

  repeat

    blockread(Source,AltIndex,sizeof(word));

    blockread(Source,NumMeasurementSets,sizeof(word));

    for I:=1 to NumMeasurementSets do

    begin

      blockread(Source,Test,sizeof(byte));

      SourceStr:='';

      for J:=1 to 10 do

      begin

        blockread(Source,Ch,sizeof(char));

        if J<=Test then SourceStr:=SourceStr+Ch;

      end;

      blockread(Source,DateTime,sizeof(TDateTime));

      blockread(Source,Latitude,sizeof(single));

      blockread(Source,Longitude,sizeof(single));

      blockread(Source,Equivlat,sizeof(single));

      blockread(Source,NumMeas,sizeof(word));

      for K:=1 to NumMeas do

      begin

        blockread(Source,Value,sizeof(single));

        blockread(Source,Error,sizeof(word));

        blockread(Source,MeasType,sizeof(byte));

      end;

    end;

  until eof(Source);

  closefile(Source);

end.


Here is some Fortran code that implements the recipe outlined above

 implicit none


      character

     & source*10,  ! Source of the set of measurements (e.g., 'SAGE1')

     & infile*100  ! BDBP file


      integer*1    ! (unsigned)

     & meastype,   ! Type of measurement (i.e., measurement ID)

     & numchar     ! Number of characters (Source name)


      integer*2    ! (unsigned*2)

     & altindex,   ! Indicator of current level

     & numset,     ! Number of measurement sets in the file

     & nummeas,    ! Number of measurements in current set

     & errp        ! Percentage error on the measurement (x100)


      real*4

     & value,      ! Value of measurement

     & lat,        ! Latitude at which the measurement set was made

     & lon,        ! Longitude at which the measurement set was made

     & eqlat       ! Equivalent latitude at which the measurement set was made


      real*8

     & datetime    ! Date and time since 12 December 1899 12:00am


      integer*4

     & i,j         ! Loop counters


!:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::


      infile='TimeSeries_01.bin'


      open(unit=7,file=trim(infile),form='binary',status='old',

&      action='read',convert='LITTLE_ENDIAN',err=990)


      do

read(7,end=10) altindex,numset

        do i=1,numset

          read(7,end=10) numchar,source,datetime,lat,lon,eqlat,nummeas

 do j=1,nummeas

            read(7,end=10) value,errp,meastype

          end do   !  end of do-while

        end do   !  end of do-while

      end do


10    continue


      close(7)


      GOTO 999


990   write(*,'2a') 'Error reading binary data from file: ',infile

      stop

999 end


Here is some IDL code that implements the recipe outlined above

pro read_bdbp_file,filename


source = ''     ; Source of the set of measurements (e.g., 'SAGE1')


meastype = 0b   ; Type of measurement (i.e., measurement ID)

numchar = 0b    ; Number of characters (Source name)


altindex = 0U   ; Indicator of current level

numset = 0U     ; Number of measurement sets in the file

nummeas = 0U    ; Number of measurements in current set

errp = 0U       ; Percentage error on the measurement (x100)


value = 0.0     ; Value of measurement

lat = 0.0       ; Latitude at which the measurement set was made

lon = 0.0       ; Longitude at which the measurement set was made

eqlat = 0.0     ; Equivalent latitude at which the measurement set was made


datetime = 0.0d ; Date and time since 12 December 1899 12:00am


;open file

openr,unit,filename,ERROR = err,/get_lun,/SWAP_IF_BIG_ENDIAN

if (err ne 0) then begin

      print,'Abort: Error opening bdbpfile: ',filename

      stop

endif


while (NOT(EOF(unit))) do begin

      readu,unit,altindex,numset

      for i=1,numset do begin

            source = '0123456789'  ; reset len(source)=10

            readu,unit,numchar,source,datetime,lat,lon,eqlat,nummeas

            for j=1,nummeas do begin

                  readu,unit,value,errp,meastype

            endfor

      endfor

end


close,unit


end

Here is some C++ code that implements the recipe outlined above

int _tmain()

{

char* Filename;

int I,J,K;

unsigned short NumMeasurementSets;

unsigned short AltIndex;

unsigned short NumMeas;

unsigned short Error;

unsigned char Test;

unsigned char MeasType;

unsigned char Ch;

double DateTime;

float Value;

float Latitude;

float Longitude;

float EquivLat;

string SourceStr;


Filename="D:/archive/BDBP_V1/LatAlt/TimeSeries_21.bin";


ifstream infile;

infile.open(Filename, ios::binary);

if (infile.is_open())

{

do

{ infile.read((char*)&AltIndex,sizeof(unsigned short));

infile.read((char*)&NumMeasurementSets,sizeof(unsigned short));

for (I=1; I<=NumMeasurementSets; I++){

infile.read((char*)&Test,sizeof(Test));

SourceStr="";

for (J=1; J<=10; J++){

infile.read((char*)&Ch,sizeof(unsigned char));

if (J<=int(Test)) {SourceStr.push_back(Ch);}

}

infile.read((char*)&DateTime,sizeof(double));

infile.read((char*)&Latitude,sizeof(float));

infile.read((char*)&Longitude,sizeof(float));

infile.read((char*)&EquivLat,sizeof(float));

infile.read((char*)&NumMeas,sizeof(unsigned short));

for (K=1; K<=NumMeas; K++){

infile.read((char*)&Value,sizeof(float));

infile.read((char*)&Error,sizeof(unsigned short));

infile.read((char*)&MeasType,sizeof(unsigned char));

}

}

} while(!infile.eof());

                infile.close();

}

return 0;

}