Programmer’s manual¶
Build system¶
Yade uses cmake the cross-platform, open-source build system for managing the build process. It takes care of configuration, compilation and installation. CMake is used to control the software compilation process using simple platform and compiler independent configuration files. CMake generates native makefiles and workspaces that can be used in the compiler environment of your choice.
Building¶
The structure of Yade source tree is presented below.
We shall call each top-level component module (excluding, doc
,
examples
and scripts
which don’t participate in the build process).
Some subdirectories of modules are skipped for brevity, see README.rst
files therein for more information:
cMake/ ## cmake files used to detect compilation requirements
core/ ## core simulation building blocks
data/ ## data files used by yade, packaged separately
doc/ ## this documentation
examples/ ## examples directory
gui/ ## user interfaces
qt5/ ## same, but for qt5
lib/ ## support libraries, not specific to simulations
preprocessing/ ## files associated with creation or generation of the simulation
dem/ ## creating a DEM simulation
potential/ ## creating a PotentialBlocks or PotentialParticles simulation
README.rst ## more information about this directory
pkg/ ## simulation-specific files
common/ ## generally useful classes
dem/ ## classes for Discrete Element Method
README.rst ## more information about this directory
postprocessing/ ## files associated with extracting results for postprocessing
dem/ ## general data extraction from DEM, no particular data target
image/ ## creating images from simulation
vtk/ ## extracting data for VTK
README.rst ## more information about this directory
py/ ## python modules
scripts/ ## helper scripts including packaging and checks-and-tests
Header installation¶
CMAKE uses the original source layout and it is advised to use #include <module/Class.hpp>
style of inclusion rather than #include "Class.hpp"
even if you are in the same directory.
The following table gives a few examples:
Original header location | Included as |
---|---|
core/Scene.hpp |
#include <core/Scene.hpp> |
lib/base/Logging.hpp |
#include <lib/base/Logging.hpp> |
lib/serialization/Serializable.hpp |
#include <lib/serialization/Serializable.hpp> |
pkg/dem/SpherePack.hpp |
#include <pkg/dem/SpherePack.hpp> |
Automatic compilation¶
In the pkg/
directory, situation is different. In order to maximally ease
addition of modules to yade, all *.cpp
files are automatically scanned recursively by
CMAKE and considered for compilation.
To enable/disable some component use the cmake flags ENABLE_FEATURE
, which are listed in:
When some component is enabled an extra #define
flag YADE_FEATURE
is passed from cmake to the compiler.
Then inside the code both the .cpp
and .hpp
files which contain the FEATURE
feature
should have an #ifdef YADE_FEATURE
guard at the beginning.
Linking¶
The order in which modules might depend on each other is given as follows:
module | resulting shared library | dependencies |
---|---|---|
lib | libyade-support.so |
can depend on external libraries, may not depend on any other part of Yade. |
core | libcore.so |
yade-support ; may depend on external libraries. |
pkg | libplugins.so |
core , yade-support |
gui | libQtGUI.so ,
libPythonUI.so |
lib , core , pkg |
py | (many files) | lib , core , pkg , external |
Development tools¶
Integrated Development Environment and other tools¶
A frequently used IDE is Kdevelop. We recommend using this software for navigating in the sources, compiling and debugging. Other useful tools for debugging and profiling are Valgrind and KCachegrind. A series of wiki pages is dedicated to these tools in the development section of the wiki.
Yade is agnostic to the IDE used; it can be compiled and run directly from the command line. You can modify the source code using any text editor, such as vim https://www.vim.org/, emacs https://www.gnu.org/software/emacs/, vscode https://code.visualstudio.com/, or any other editor of your choice.
Hosting and versioning¶
The Yade project is kindly hosted at Launchpad and GitLab:
- source code on gitlab
- issue and bug tracking on gitlab
- release downloads on GitLab
- yade-dev mailing list on launchpad: yade-dev@lists.launchpad.net
- yade-users mailing list on launchpad: yade-users@lists.launchpad.net
- questions and answers on GitLab
The versioning software used is GIT, for which a short tutorial can be found in Yade on GitLab. GIT is a distributed revision control system. It is available packaged for all major linux distributions.
The source code is periodically imported to Launchpad for building PPA-packages. The repository can be http-browsed.
Development process¶
Git is used for version control. The main development branch is called master
and is hosted at GitLab.
For the development process, the following steps are recommended:
- Clone the repository to your local machine:
git clone https://gitlab.com/yade-dev/trunk.git
- Create a new branch for your work:
git checkout -b my-new-feature
- Make your changes and commit them:
git commit -am 'Add some feature'
- Push to the branch:
git push origin my-new-feature
- Submit a merge request on GitLab: Merge Request
The merge request will be reviewed by the developers and, if accepted, merged into the main branch. Yade has a wide range of pipelines that are automatically triggered by GitLab when a new commit is pushed to the repository. These pipelines include building the software, running tests, and generating the documentation. The results of these pipelines can be viewed on the GitLab CI/CD page or by clicking on the green checkmark next to a commit in the GitLab interface. If some tests fail, the developers will be notified and the merge request will not be accepted until the issues are resolved.
It is required to add at least one line into the ChangeLog file in the root directory of the repository for each merge request. This file is used to generate the release notes for each new version of Yade.
How to make a release¶
The release process is automated using GitLab CI/CD pipelines. The release process is triggered by creating a new tag in the repository.
The tag should be named according to the version number, e.g. 2022.01a
.
The release process will build the software, run tests, and generate the documentation.
Create RELEASE file in the root folder with the version number in it.
Add new changelog entries to Changelog using “git shortlog PREVVERSION..”.
Create branch using the following command and format:
git checkout -b YYYY.MM
Tag release “git tag -as YYYY.MMa -m”YYYY.MMa”
Return to master branch and remove RELEASE file
Push master, new branch and tags to gitlab
Download tar.gz
Create asc-file (signature):
gpg --armor --sign --detach-sig tarball.tar.gz
Upload new tarball on Launchpad
Make announcement on mailing list and on Launchpad.
RELEASE file should contain the version number in the following format:
YYYY.MM
where YYYY
is the year and MM
is the month of the release. For example, the release file for the January 2022 release should contain the following text:
2022.01a
Build robot¶
A build robot hosted at UMS Gricad is tracking source code changes via gitlab pipeline mechanism. Each time a change in the source code is committed to the main development branch via GIT, or a Merge Request (MR) is submitted the “buildbot” downloads and compiles the new version, and then starts a series of tests.
If a compilation error has been introduced, it will be notified to the yade-dev mailing list and to the committer, thus helping to fix problems quickly. If the compilation is successful, the buildbot starts unit regression tests and “check tests” (see below) and report the results. If all tests are passed, a new version of the documentation is generated and uploaded to the website in html and pdf formats. As a consequence, those two links always point to the documentation (the one you are reading now) of the last successful build, and the delay between commits and documentation updates are very short (minutes). The buildbot activity and logs can be browsed online.
The output of each particular build is directly accessible by clicking the green “Passed” button, and then clicking “Browse” in the “Job Artifacts” on the right.
Debugging¶
For yade debugging two tools are available:
Use the debug build so that the stack trace provides complete information about potential crash. This can be achieved in two ways:
- Compiling yade with cmake option
-DDEBUG=ON
, - Installing
yade-dbgsym
debian/ubuntu package (this option will be available after this task is completed).
- Compiling yade with cmake option
Use Logging framework described below.
These tools can be used in conjunction with other software. A detailed discussion of these is on yade wiki. These tools include: kdevelop, valgrind, alleyoop, kcachegrind, ddd, gdb, kompare, kdiff3, meld.
Note
On some linux systems stack trace will not be shown and a message ptrace: Operation not permitted
will appear instead. To enable stack trace issue command: sudo echo 0 > /proc/sys/kernel/yama/ptrace_scope
. To disable stack trace issue command sudo echo 1 > /proc/sys/kernel/yama/ptrace_scope
.
Hint
When debugging make sure there is enough free space in /tmp.
Logging¶
Yade uses boost::log library for flexible logging levels and per-class debugging.
See also description of log module.
A cmake compilation option -DENABLE_LOGGER=ON
must be supplied during compilation [1].
Figure imgLogging shows example use of logging framework. Usually a ClassName
appears in place of _log.cpp
shown on the screenshot. It is there because the yade.log
module uses CREATE_CPP_LOCAL_LOGGER
macro instead of the regular DECLARE_LOGGER
and CREATE_LOGGER
, which are discussed below.
Note
Default format of log message is:
<severity level> ClassName:LineNumber FunctionName: Log Message
special macro LOG_NOFILTER
is printed without ClassName
because it lacks one.
Config files can be saved and loaded via readConfigFile and saveConfigFile. The defaultConfigFileName is read upon startup if it exists. The filter level setting -f
supplied from command line will override the setting in config file.
[1] | Without -DENABLE_LOGGER=ON cmake option the debug macros in /lib/base/Logging.hpp use regular std::cerr for output, per-class logging and log levels do not work. |
Log levels¶
Following debug levels are supported:
macro name | filter name | option | explanation |
---|---|---|---|
LOG_NOFILTER |
log.NOFILTER |
-f0 |
Will print only the unfiltered messages. The LOG_NOFILTER macro is for
developer use only, so basically -f0 means that nothing will be printed.
This log level is not useful unless a very silent mode is necessary. |
LOG_FATAL |
log.FATAL |
-f1 |
Will print only critical errors. Even a throw to yade python interface will not recover from this situation. This is usually followed by yade exiting to shell. |
LOG_ERROR |
log.ERROR |
-f2 |
Will also print errors which do not require to throw to yade python interface. Calculations will continue, but very likely the results will be all wrong. |
LOG_WARN |
log.WARN |
-f3 |
Will also print warnings about recoverable problems that you should be notified about (e.g., invalid value in a configuration file, so yade fell back to the default value). |
LOG_INFO |
log.INFO |
-f4 |
Will also print all informational messages (e.g. something was loaded, something was called, etc.). |
LOG_DEBUG |
log.DEBUG |
-f5 |
Will also print debug messages. A yade developer puts them everywhere, and yade user enables them on per-class basis to provide some extra debug info. |
LOG_TRACE |
log.TRACE |
-f6 |
Trace messages, they capture every possible detail about yade behavior. |
Yade default log level is yade.log.WARN
which is the same as invoking yade -f3
.
Setting a filter level¶
Warning
The messages (such as a << b << " message."
) given as arguments to LOG_*
macros are used only if the message passes the filter level. Do not use such messages to perform mission critical calculations.
There are two settings for the filter level, the Default
level used when no ClassName
(or "filename.cpp"
) specific filter is set and a filter level set for specific ClassName
(or "filename.cpp"
). They can be set with following means:
- When starting yade with
yade -fN
command, whereN
sets theDefault
filter level. The default value isyade.log.WARN
(3). - To change
Default
filter level during runtime invoke commandlog.setLevel("Default",value)
orlog.setDefaultLogLevel(value)
:
Yade [1]: import log
Yade [2]: log.setLevel("Default",log.WARN)
Yade [3]: log.setLevel("Default",3)
Yade [4]: log.setDefaultLogLevel(log.WARN)
Yade [5]: log.setDefaultLogLevel(3)
- To change filter level for
SomeClass
invoke command:
Yade [6]: import log
Yade [7]: log.setLevel("NewtonIntegrator",log.TRACE)
Yade [8]: log.setLevel("NewtonIntegrator",6)
- To change the filter level for
"filename.cpp"
use the name specified when creating it. For example manipulating filter log level of"_log.cpp"
might look like following:
Yade [9]: import log
Yade [10]: log.getUsedLevels()
Out[10]: {'Default': 3, 'NewtonIntegrator': 6}
Yade [11]: log.setLevel("_log.cpp",log.WARN)
Yade [12]: log.getUsedLevels()
Out[12]: {'Default': 3, 'NewtonIntegrator': 6, '_log.cpp': 3}
Yade [13]: log.getAllLevels()["_log.cpp"]