GSoc 2015 Week 12 & 13

This week we announced the release of SymEngine on Sage list. For that, I made some changes into the build system for versioning and to use SymEngine from other C/C++ projects.

First, SymEngineConfig.cmake would output a set of flags, imported dependencies, etc. SymEngineConfigVersion.cmake would check that the version is compatible and if the 32/64-bitness is correct of the SymEngine project and the other CMake project. When SymEngine is only built, then these files would be at the root level and when installed they would be at /lib/cmake/symengine. An excerpt from the wiki page, I wrote at, https://github.com/sympy/symengine/wiki/Using-SymEngine-from-a-Cpp-project

Using SymEngine in another CMake project

To use SymEngine from another CMake project include the following in yourCMakeLists.txt file

find_package(SymEngine 0.1.0 CONFIG)

You can give the path to the SymEngine installation directory if it was installed to a non standard location by,

find_package(SymEngine 0.1.0 CONFIG PATHS /path/to/install/dir/lib/cmake/symengine)

Alternatively, you can give the path to the build directory.

find_package(SymEngine 0.1.0 CONFIG PATHS /path/to/build/dir)

An example project would be,

cmake_minimum_required(VERSION 2.8)
find_package(symengine 0.1.0 CONFIG)
set(CMAKE_CXX_FLAGS_RELEASE ${CMAKE_CXX_FLAGS_RELEASE} "-std=c++0x")

include_directories(${SYMENGINE_INCLUDE_DIRS})
add_executable(example main.cpp)
target_link_libraries(example ${SYMENGINE_LIBRARIES})

More options are here

Using SymEngine in Non CMake projects

You can get the include flags and link flags needed for SymEngine using the command line CMake.

compile_flags=`cmake --find-package -DNAME=SymEngine -DCOMPILER_ID=GNU -DLANGUAGE=CXX -DMODE=COMPILE`
link_flags=`cmake --find-package -DNAME=SymEngine -DCOMPILER_ID=GNU -DLANGUAGE=CXX -DMODE=LINK`

g++ $compile_flags main.cpp $link_flags

Python wrappers

There was a suggestion to make the Python wrappers separate, so that in a distribution like Gentoo, the package sources can be distributed separately.

So, I worked on the Python wrappers to get them to be built independently or with the main repo. Now, the python wrappers directory along with the setup.py file from the root folder can be packaged and they would work without a problem.

GSoC week 10 and 11

symengine-0.1.0 beta version was released this week and these two weeks were spent on making sure symengine works without a problem on Sage.

One issue was the linking of the python libraries in Sage. In binary releases of sage, the variable distutils.sysconfig.get_config_var('LIBDIR')  is wrong. It is set to the build machine's location. In Windows this is set to empty. Earlier, to link the python libraries into the python wrappers, python library was found using the above variable, but in some cases like Sage and Windows this method fails. To fix this, CMake now looks in `sys.prefix`/libs and `sys.prefix`/lib as well to find the python libraries.

Another issue that came up was cmake generating bad link flags. When installing in Sage, it is important to make sure the libraries in Sage are linked and not the system wide libraries. To do that libraries were searched for in the sage directories ignoring the system wide libraries. When given the full path of the libraries to link, we noticed a strange behaviour. `/path/to/sage/local/lib/libgmp.so` was changed to `-lgmp` causing the linker to pick up the system-wide gmp library. 

After reading through CMake documentation, I realized that this was due to `find_library` giving wrong locations of system libraries where there are multiple libraries of the same name for different architectures. For example if the output of `find_library(math NAMES m)` was given to find the standard math library, it may find a `libm.so` that was intended for a different architecture. Therefore when cmake realizes that the library being linked to is a system library then the full path is converted to `-lm` to delegate the task to the linker to find the correct library. 

This behaviour is useful for some scenarios, but in our case, this was not the behaviour I needed. Fortunately there was a workaround for this mentioned in the documentation. Using IMPORTED target feature in CMake, I was able to get CMake to use the full path of the library.

R7 and R8 benchmarks from symbench benchmarks of sage were added to benchmark SymEngine-C++ against GiNaC and also SymEngine-Python against SymPy and Sage.

GSoC Week 8 and 9

These two weeks me and Ondrej started adding support for different compilers.

I added support for MinGW and MinGW-w64. There were some documented, but not yet fixed bugs in MinGW that I encountered. When including cmath, there were errors saying `_hypot` not defined, and `off64_t` not defied. I added flags `-D_hypot=hypot -Doff64_t=_off64_t` to fix this temporarily. With that symengine was successfully built.

For python wrappers in windows, after building there was a wrapper not found error which was the result of not having the extension name as pyd in windows. Another problem faced was that, python distribution's `libpython27.a` for x64 was compiled for 32 bit architecture and there were linking errors. I found some patched files at http://www.lfd.uci.edu/~gohlke/pythonlibs/#libpython and python wrappers were built successfully. Also added continuous integration for MinGW using appveyor.

With MinGW, to install gmp all you had to do was run the command `mingw-get install mingw32-gmp`. For MinGW-w64, I had to compile gmp. For this appveyor came in handy. I started a build in appveyor, stopped it and then logged into the appveyor machine remotely using `remmina` (Each VM was shutdown after 40 minutes. Within that 40 minutes you can login and debug the building). I compiled gmp using msys and mingw-w64 and then downloaded them to my machine. For appveyor runs, these pre-compiled binaries of gmp were used to test MinGW-w64

Ondrej and I worked together to make sure SymEngine could be built using MSVC in Debug mode. Since gmp couldn't be used out of the box in MSVC, we used MPIR project's sources which included visual studio project files. MPIR is a fork of GMP and provides MSVC support. We used it to build SymEngine in MSVC. Later I added support for Release mode and also added continuous integration for both build types and platform types.

Python extension can also be built with MSVC. We are testing the Python extensions in Release mode only right now, because appveyor has only python release mode libraries and therefore when building the extension in Debug mode it gives an error saying python27_d.lib is not found.

I also improved the wrappers for Matrix by adding `__getitem__` and `__setitem__` so that the matrices can be used easily in Python.

Another improvement to SymEngine was the automatic simplification of expressions like `0.0*x` and `x**0.0`. These expressions are not simplified more in master, so I 'm proposing a patch to simplify them to `0.0` and `1.0` respectively.

GSoC Week 7

This week I worked on the Sage wrappers and Python wrappers. To make it easier to try out symengine, I made changes to the sage wrappers such that if sage does not have symengine_conversions methods, (i.e. sage not updated to the symengine branch) then conversions would be done via python strings. For example, an integer is converted to a Python string and then to a Sage integer. This is slow, but makes it easier to install symengine. You can try it out by downloading cmake-3.2.3.spkg and symengine-0.1.spkg and installing them. (Link to download is .....) To install type

sage -i /path/to/cmake-3.2.3.spkg

sage -i /path/to/symengine-0.1.spkg

Python wrappers included only a small amount of functions from SymEngine. Wrappers were added to functions like log, trigonometric functions, hyperbolic functions and their inverses.

CMake package for Sage is now ready for review, http://trac.sagemath.org/ticket/18078.

SymEngine package for Sage can be found here, https://github.com/isuruf/sage/tree/symengine. A PR would be sent as soon as CMake ticket is positively reviewed.

Next week, testing with Sage, Python docstrings, SymEngine package for Sage are the main things that I have planned for now. Also a PyNumber class to handle python numbers would be started as well.

GSoC Week 6

This week, I worked on improving the testing and making Sage wrappers. First, building with Clang had several issues and they were not tested. One issue was a clang bug when `-ffast-math` optimization is used. This flag would make floating point arithmetic perform better, but it may do arithmetic not allowed by the IEEE floating point standard. Since it performs faster we have enabled it in Release mode and due to a bug in clang, a compiler error is given saying  error: unknown type name '__extern_always_inline' . This was fixed by first checking if the error is there in cmake and then adding a flag D__extern_always_inline=inline. Another issue was that type_traits header was not found. This was fixed by upgrading the standard C++ library, libstdc++

This week, I finished the wrappers for Sage. Now converters to and from sage can be found at sage.symbolic.symengine. For this module to convert using the C++ level members, symengine_wrapper.pyx 's definitions of the classes were taken out and declared in symengine_wrapper.pxd and implemented in pyx file. To install symengine in sage, https://github.com/sympy/symengine/issues/474 has to be resolved. A cmake check will be added to find whether this issue exists and if so, then the flag -Wa,-q will be added to the list of flags. We have to make a release of symengine if we were to make spkg's to install symengine in Sage, so some of my time next week will involve getting symengine ready for a release and then making spkgs for everyone to try out symengine.

GSoC Week 5

This week, I looked into installing SymEngine in Sage and wrappers. One issue was that we have a header named `complex.h` and some libraries look inside `$SAGE_LOCAL/include` for the C header `complex` and this leads to errors. It was decided to install symengine headers inside a folder called `symengine` so that a header could be accessed by `symengine/basic.h` form and it avoids clashes.

I looked at other libraries for how this is done. Some libraries have the installing headers in a separate folder like `include/project_name`, but `.cpp`s under `src`. Some libraries have headers and source files in the same folder and they are included in `project_name` folder. Since SymEngine has headers and sources in the same folder, we decided to rename `src` to `symengine`. This lead to another problem. Python wrappers folder was named `symengine`. So I moved them to `symengine/python/symengine` to enable using symengine python wrappers inside the build directory (although you have to change directory into `symengine/python` to use it).

Some other work involved, making sure `make install` installed all the python tests and installing dependencies in Travis-ci without using sudo.

That's all the updates for this week. Mid evaluations are coming this week, so I hope to get a SymEngine spkg done this week.

GSoC 2015 - Week 4

This week I got access to OS X, so I decided to do all the work related to OS X this week while I had access. First of all, I worked on getting CMake to build on Sage in OS X 10.10. CMake is supported to be built using clang on OS X and is not supporting gcc. Since sage uses gcc for building packages, I tried building CMake on Sage. (Thanks to +Rajith for giving me the chance to work on his Mac).

Main problem with CMake in OSX 10.10 was that it uses an Apple header <CoreFoundation/CoreFoundation.h> which is a collection of headers including <CoreFoundation/CFStream.h> which in turn includes a faulty Apple header '/usr/local/include/dispatch/dispatch.h'. After going through CMake code, it seemed that although the header 'CoreFoundation.h' was included 'CFStream.h' was not needed. So I used the specific headers needed (<CoreFoundation/CFBundle.h> etc.) and CMake was successfully built on Sage. Testing the CMake installation resulted in 6 out of 387 tests failing.

Another good news was that we got access to test SymEngine on TravisCI with OSX. We are testing clang and gcc both to make sure, symengine builds on both. Building with clang was successful, but with gcc there were couple of problems and was hard to check on TravisCI as there were huge queuing times for builds on OSX.

One issue is that on OSX, gmp library we are linking to is installed by homebrew. g++ was using a different C++ standard library than what gmp was compiled with and hence linking errors occur. A CMake check was added to try to compile a simple program with gmpxx and if it fails, give an error message at configuration time.

Another issue was that `uint` was used in some places instead of `unsigned int`. On Linux and OSX clang `unsigned int` was typedef as `uint`, so there was no problem detected in automated tests in Travis-CI. Since `uint` is not a C++ standard type, it was changed to `unsigned int`.

Next week, I will try to figure out why 6 tests in CMake test suite fails and try to fix those and get CMake into optional packages. Also I will work on the wrappers for sage for SymEngine.