Documentation to use data

1. Select type of geometry

deus-library_index

Comoving Space

Particles and halos at a given time

Redshift Space

Particles and halos as seen by an observer

2. Select your simulation

What are the simulations

Today, only DEUSS and DEUS FUR simulations are available through the web. Data from other DEUS Consortium projects as DEUS PUR, will be available later on.

DEUSS consists in 5 N-body simulations with 2048^3 particles, 9 simulations with 1024^3 particles (plus some 512^3 and 256^3 particles simulations for tests or convergence studies). We have explored three cosmologies: LCDM WMAP5 (lcdmw5), quintessence with Ratra-Peebles potential (rpcdmw5) and quintessence with Supergravity corrections (sucdmw5). Box lengthes are 2592 Mpc/h, 648 Mpc/h and 162 Mpc/h. All the simulations share the same realization of the initial density field and have been run with the same code: RAMSES, modified for quintessence.

DEUS-FUR (Full Universe Run) are 3 N-body simulations with 8192^3 particles and a box length of 21000 Mpc/h. The cosmologies explored are: LCDM WMAP7 (lcdmw7), quintessence with Ratra-Peebles potential (rpcdmw7) and ghost model with a constant equation of state (wcdmw7).

The goal of this simulation suite is to investigate the imprints of dark energy on cosmic structure formation at various scales from Gpc to kpc and for various realistic cosmologies calibrated on CMB and SNIa.

More informations about the simulations are available on the site.

How to select one

The simulations are sorted in a table by box length and cosmology. In each cell, you’ll find the available resolution for the combination.
Just click on the one you want to select it.

deus-library_1_simulation

When selected, the simulation properties appears below as well as the available snapshots.

deus-library_2_snapshot

When a snapshot is selected, his own properties appear below the simulation ones as well as the available objects and their formats.

deus-library_3_file

When an object is select, the corresponding files appear to be downloaded.
deus-library_4_download

It is both possible to download the file list as a text file to use with wget, or to download files individually

3. File Formats

Types of files

Fields (particles)

What is it? Particles sorted per region

What are the particles properties? Position (distribution), velocity (kinematics), identity (follow particles from one snapshot to another)

What are the redshifts available? More than 10 per simulations:
a=astart,0.025,0.05,0.1,0.15…,0.95,1

What are the scales probed? From 3. kpc to 30. Gpc

What could I do with it? Any field oriented analysis: correlation, distribution, filament, weak lensing, BAO…

Halos

What is it? Particles sorted on a per halo basis

What are the particles properties? Position, velocity, identity …

What are the redshifts available? More than 10 per simulations:
a=astart,0.025,0.05,0.1,0.15…,0.95,1

What are the halos probed? From 2.5 .10^10 to 1. 10^16 Msun, from 100 to 3 millions particles.

What could I do with it? Any halo oriented analysis: cluster counts, strong lensing, statistics on macroscopic properties and internal profiles…

Format of the files

The Field and Halo files are binary big endian unformated Fortran files for lcdmw5, rpcdmw5, sucdmw5 cosmologies and binary little endian unformated Fortran files for other cosmologies.

They can be read natively in Fortran, or using a provided library in C++ as described below.
Libraries for other languages will be provided later on, some tips are given below to create yours.

Units are ramses units (you can use unit_t, unit_l, unit_d to convert to physical cgs units).

Field file description (particles)

npart integer(4) Number of particles in the field
procid integer(4) ID of the cube
boundaries real(4)*6 xmin,xmax,ymin,ymax,zmin,zmax
positions real(4)*3*npart x,y,z position of each particles
velocities real(4)*3*npart vx,vy,vz velocity of each particles
ids int(8)*npart unique id of each particle

Halo position file description

nhalos integer(4) Number of halos in the file
positions (int(8)+real(4)*3+int(4))*npart id,x,y,z,npart of each halo

Halo particles file description

nhalos integer(4) Number of halos in the file

Then, for each halo:

npart integer(4) Number of particles in the field
positions real(4)*3*npart x,y,z position of each particles
velocities real(4)*3*npart vx,vy,vz velocity of each particles
ids int(8)*npart unique id of each particle

How to read the file using Fortran

The different files are created using Fortran, so they can easily be read using this language.

However, be careful that they have been generated from Big Endian supercomputers
so you need to use the proper compiler setting to read them correctly.
For instance, using ifort:

       ifort -convert big_endian -O3 my_program.f90 -o my_program

Read Field file in Fortran

Program sample to read a field file:

program read_field
    implicit none

    integer::mynpart,procID,Ucub,j,i
    real::xmin,xmax,ymin,ymax,zmin,zmax
    integer(8),allocatable::id(:)
    real,allocatable::x(:,:),v(:,:)

    Open(Unit=Ucub,file='/path_for_data/fof_boxlen648_n1024_lcdmw5_cube_00001',Form='Unformatted')
    ! read number of particles integer*4
    Read(Ucub) mynpart
    ! read ID of the FoF process assigned to this cube integer*4
    Read(Ucub) procID
    ! read boundary values of the cube real*4
    Read(Ucub) xmin,xmax,ymin,ymax,zmin,zmax
    ! allocate memory
    allocate(id(1:mynpart))
    allocate(x(1:3,1:mynpart))
    allocate(v(1:3,1:mynpart))
    ! read positions real*4
    Read(Ucub) ((x(j,i),j=1,3),i=1,mynpart)
    ! read velocities real*4
    Read(Ucub) ((v(j,i),j=1,3),i=1,mynpart)
    ! read id's integer*8 WARNING
    Read(Ucub) (id(i),i=1,mynpart)
    Close(Ucub)

end program read_field

Read Halo file in Fortran

Program sample to read the 1st halo of a halo file:

 program read_struct
    implicit none

    integer::mynpart,procID,Ucub,j,i,mynhalos
    real::xmin,xmax,ymin,ymax,zmin,zmax
    integer(8),allocatable::id(:)
    real,allocatable::x(:,:),v(:,:)

    Open(Unit=Ucub,file='/path_for_data/fof_boxlen648_n1024_lcdmw5_strct_00001',Form='Unformatted')
    ! read number of halos integer*4
    Read(Ucub) mynhalos

    ! READ 1st HALO ONLY
    ! read number of particles integer*4
    Read(Ucub) mynpart
    ! allocate memory
    allocate(id(1:mynpart))
    allocate(x(1:3,1:mynpart))
    allocate(v(1:3,1:mynpart))
    ! read positions real*4
    Read(Ucub) ((x(j,i),j=1,3),i=1,mynpart)
    ! read velocities real*4
    Read(Ucub) ((v(j,i),j=1,3),i=1,mynpart)
    ! read id's integer*8 WARNING
    Read(Ucub) (id(i),i=1,mynpart)

    Close(Ucub)

end program read_struct

How to read the file using C++

For C++, a special library has been created, FOFReaderLib that you can find on GitHub:

https://github.com/pasdeloup/FOFReaderLib

Read Field file in C++

Program sample to read the a field file and display some infos about it:

#include "FOFReaderLib/FOFReaderLib.h"

int main(void)
{
    // Simple line to read a Field. The second arg is to ask reading the ids (default is false)
    FOFCube cube("/path_for_data/fof_boxlen648_n1024_lcdmw5_cube_00001", true);

    // Show some details about the read cube as an example of usage
    std::cout
    << cube.npart() << " particles, "
    << "area: (" << cube.minX() << "," << cube.minY() << "," << cube.minZ() << ") "
    << "to (" << cube.maxX() << "," << cube.maxY() << "," << cube.maxZ() << ")"
    << std::endl;
    for(int j=0; j< std::min(10,cube.npart()); j++) {
    std::cout << " Particle " << j << ": "
    << "id: " << cube.id(j) << " "
    << "position: (" << cube.posX(j) << "," << cube.posY(j) << "," << cube.posZ(j) << ") "
    << "velocity (" << cube.velX(j) << "," << cube.velY(j) << "," << cube.velZ(j) << ")"
    << std::endl;
    }
return 0;
}

Read Halo file in C++

Program sample to read the a halo file and display some infos about each halo:

#include "FOFReaderLib/FOFReaderLib.h"

int main(void)
{
    // Simple line to read a list of halos
    FOFStrct strct("/path_for_data/fof_boxlen648_n1024_lcdmw5_strct_00001");

    // List the halos from the file as an example of usage
    for(int i=0; i
    npart() << " particles"
    << std::endl;
    }
    return 0;
}

How to read the file using other languages

To read the files in other languages, you need to be able to read Fortran binary files, and to manage Big Endianess.

Read Fortran binary files

In Fortran, each Write command add a record marker before and after the data. This record marker is a 4 byte
integer giving the number of bytes wrotten.

For instance, to read the (x,y,z) coordinates of 200 particles, you will find 200*3*4 + 2*4 = 2408 bytes:

    4 bytes: head record marker, integer = 2400
    4 bytes: x coordinate of the 1st particle, float
    4 bytes: y coordinate of the 1st particle, float
    4 bytes: z coordinate of the 1st particle, float
    4 bytes: x coordinate of the 2nd particle, float
    4 bytes: y coordinate of the 2nd particle, float
    4 bytes: z coordinate of the 2nd particle, float
    (...)
    4 bytes: x coordinate of the 200th particle, float
    4 bytes: y coordinate of the 200th particle, float
    4 bytes: z coordinate of the 200th particle, float
    4 bytes: foot record marker, integer = 2400

Read Big Endian Files

Bytes can be wrotten either in Big Endian (BE) or Little Endian (LE) on a computer, LE being the most common.
http://en.wikipedia.org/wiki/Endianness

To read BE files on a LE computer, some languages like Fortran allow to easily switch using a compiler
option, or have special functions.

If you don’t have, you’ll need to manually swap the order of the bits composing each byte. That’s what is
done in the C++ FOFReaderLib library.

How to visualize the files

To visualize field and halos in real time, a Paraview plugin will soon be
available.

Be careful that a powerful GPU is needed for fields or big halos.

For raytracing images, a custom version of Splotch has been created and can be downloaded on : https://github.com/deusconsortium/splotch