QGIS development on macOS

APRIL 22, 2020 macos qgis

We introduced a new packaging and build environment for QGIS and its dependencies for macOS. This will bring a completely new experience for QGIS macOS developers and subsequently, the users.

Overview of the changes

What is new

In 2019, we upgraded the QGIS macOS packaging to address several issues at the time. This was made possible with the help of the QGIS project and donors who supported our work.

The upgraded system had a more transparent and automated packaging approach. But it came with its own limitation. We have been using Homebrew to fetch the QGIS dependencies and to compile only QGIS for the LTR, PR and nightly releases. The main problems with this approach are:

  • Homebrew can only support one version of a package. This will limit QGIS to be built against multiple versions of, for example GDAL or Proj libraries. In one hand, we want QGIS nightlies to use more bleeding edge versions of the dependencies (e.g. for plugin or core developers) . On the other hand, using a more stable and tested versions of the dependencies for PR or LTR are not recommended (e.g. users and organisations). OSGeo4W offers the same approach for different versions of QGIS in Windows. Unfortunately, Homebrew does not offer such an option.

  • Continuous integration has been a major part of QGIS infrastructure to ensure pull requests by developers do not break workflows in certain Operating Systems. Microsoft Windows and Linux have been well supported in the CI. But with the macOS packages, it was left to user to test and report back any bugs after Pull Requests were merged to QGIS source code.

With the recent work, we have addressed both of the above issues. QGIS and almost all of its dependencies are built from sources. It will be also possible to easily integrate the code with the current CI for macOS with GitHub Workflows. This will bring more stability and control over the way we build and package QGIS for our users, whilst helping developers to identify issues with their pull requests for macOS platform, before merging them to the QGIS source code.

QGIS development on macOS

If you want to check the latest QGIS master on macOS or tweak the code to see some new exciting features, now it should be easier with the following steps, to compile QGIS from source code an a clean machine:

1. Install XCode

  • Upgrade to latest MacOS version
  • Install latest XCode from the official AppStore
  • Open XCode and accept license
  • Install command line tools from XCode, AppStore or from command line by sudo xcode-select --install

2. Install Qt and QGIS-Deps

  • Download the latest install qgis-deps script and check the version of Qt required
  • Install the required Qt version referenced from the official Qt download area. Or alternatively download qt-version.tar.gz from deps area
  • Download latest qgis-deps package too.
  • Run install_qgis_deps.bash script to install Qt in /opt/Qt/5.14.1/clang_64 and QGIS-Deps to /opt/QGIS/qgis-deps-0.3.0 folder. You may need to use root privileges or create those folders under /opt/ and assign the right permission beforehand.

3. Install other tools

  • Install Homebrew
  • Install minimal set of packages required for macOS build tools listed in this list

You may use MacPorts, Conda or other packaging system, but by the end you need cmake, git, astyle and other useful tools to be on the system PATH

4. Download and compile QGIS

  • Open the terminal (Note that the qgis-deps package is not yet signed, so you may need to add Terminal to System Preferences -> Security & Privacy -> Privacy -> Developer Tools)
  • In your work folder (e.g. ~/Projects/), clone QGIS with git clone git@github.com:qgis/QGIS.git
  • Create a build folder mkdir -p ~/Projects/build-QGIS
  • Go to the created empty directory cd ~/Projects/build-QGIS
  • Run CMake to generate the build system (use your download qgis-deps and qt package) 
    QGIS_DEPS_VERSION=0.3.0;\
QT_VERSION=5.14.1;\
PATH=/opt/QGIS/qgis-deps-${QGIS_DEPS_VERSION}/stage/bin:$PATH;\
cmake -DQGIS_MAC_DEPS_DIR=/opt/QGIS/qgis-deps-${QGIS_DEPS_VERSION}/stage \
    -DCMAKE_PREFIX_PATH=/opt/Qt/${QT_VERSION}/clang_64 \
    ../QGIS

Note that all libraries are picked from qgis-deps and not from system /usr/lib or Homebrew’s /usr/local/ or system Frameworks /Library/Frameworks/. Especially check Proj, GDAL, sqlite3 and Python paths. You should see output similar to this:

-- QGIS version: 3.13.0 Master (31300)
-- Found OpenCL: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/OpenCL.framework (found version "1.2")
-- Found OpenCL C++ headers: /Users/peter/Projects/mesh/QGIS/external/opencl-clhpp/include
-- Found GRASS 7: /opt/QGIS/qgis-deps-0.3.0/stage/grass78 (7.8.2, off_t size = 8)
-- Looking for openpty
-- Looking for openpty - found
-- Found Proj: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libproj.dylib version 6 (6.3.1)
-- Found GEOS: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libgeos_c.dylib (3.8.1)
-- Found GDAL: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libgdal.dylib (3.0.4)
-- Found Expat: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libexpat.dylib
-- Found Spatialindex: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libspatialindex.dylib
-- Found Qwt: /opt/QGIS/qgis-deps-0.3.0/stage/lib/qwt.framework (6.1.4)
-- Found LibZip: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libzip.dylib
-- Found libzip: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libzip.dylib
-- Found Sqlite3: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libsqlite3.dylib
-- Found Protobuf: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libprotobuf.dylib (found version "3.11.4")
-- Found Protobuf: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libprotobuf.dylib
-- Found ZLIB: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libz.dylib (found version "1.2.11")
-- Found zlib: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libz.dylib
-- Found PostgreSQL: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libpq.dylib
-- Found SpatiaLite: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libspatialite.dylib
-- Qt WebKit support enabled
-- Found Qt version: 5.14.1
-- Found QScintilla2: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libqscintilla2_qt5.dylib (2.11.4)
-- Found QtKeychain: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libqt5keychain.dylib
-- Found QCA: /opt/QGIS/qgis-deps-0.3.0/stage/lib/qca-qt5.framework (2.3.0)
-- Found QCA OpenSSL plugin
-- Found Libtasn1: /opt/QGIS/qgis-deps-0.3.0/stage/include
-- Pedantic compiler settings enabled
-- Found PythonInterp: /opt/QGIS/qgis-deps-0.3.0/stage/bin/python3 (found suitable version "3.7.7", minimum required is "3")
-- Found Python executable: /opt/QGIS/qgis-deps-0.3.0/stage/bin/python3
-- Found Python version: 3.7.7
-- Found Python library: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libpython3.7m.dylib
-- Found Python site-packages: /opt/QGIS/qgis-deps-0.3.0/stage/lib/python3.7/site-packages
-- Found PyQt5 version: 5.14.1
-- Found SIP version: 4.19.21
-- Found QScintilla2 PyQt module: 2.11.4
-- txt2tags not found - disabled
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- Found exiv2: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libexiv2.dylib
-- HDF5: Using hdf5 compiler wrapper to determine C configuration
-- Found HDF5: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libhdf5.dylib;/opt/QGIS/qgis-deps-0.3.0/stage/lib/libz.dylib;/usr/lib/libdl.dylib;/usr/lib/libm.dylib (found version "1.10.0")
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.29.2")
-- Found NetCDF: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libnetcdf.dylib
-- Found LibXml2: /opt/QGIS/qgis-deps-0.3.0/stage/lib/libxml2.dylib (found version "2.9.10")
-- Looking for updwtmpx
-- Looking for updwtmpx - not found
-- Found GSL: -L/opt/QGIS/qgis-deps-0.3.0/stage/lib -lgsl -lgslcblas
-- Using PROJ 6 srs database.
-- Ctest Binary Directory set to: /Users/peter/Projects/mesh/build-QGIS/output/bin
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/peter/Projects/mesh/build-QGIS
  • Compile QGIS make -j4 and wait for [100%] mark :)
  • Now you can run QGIS with command ./output/bin/QGIS.app/Contents/MacOS/QGIS or open ./output/bin/QGIS.app

Notes:

  • If you use QT Creator, you can open already generated project to modify files from it.
  • The critical CMake variable is QGIS_MAC_DEPS_DIR and you must ensure it contains valid qgis-deps installation
  • You can add more variables to CMake command to tweak the build settings, e.g. enable 3D with ‘-DWITH_3D’, etc.
  • Consult QGIS’s [INSTALL]https://github.com/qgis/QGIS/blob/master/INSTALL) for latest developement tips

Development of the QGIS-Deps package

If you want to help with development of the qgis packages on macOS, please feel free to contact peter.petrik@lutraconsulting.co.uk. The development is taking place on public QGIS-Mac-Packager repository.

Next steps

Currently, we are carrying out the following tasks:

  • to move all-in-one QGIS bundle for LTR/PR/nighlies to use this new package of dependencies. Check out Twitter for for any news on the packages. We are aiming to have the new packaging in place for the QGIS 3.14 release, which will bring GDAL 3 and Proj 6 to macOS users

  • to sign the qgis-deps packages, so that developers do not need to tweak Privacy settings to use it for compiling QGIS and its dependencies.

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Posted by Peter Petrik
← Previous Next →