Limit CPU cores for ndkBuild with Cmake and Ninja

Question

Before, when I was using ndkBuld for building native code on Android I was able to pass an argument to make to define a number of CPU cores to be used. If I wanted to utilize 4 cores I could add something like this

externalNativeBuild {
    ndkBuild {
        arguments "-j4", "APP_SHORT_COMMANDS=true"
        abiFilters "armeabi-v7a"
    }
}

Can somebody give an advice how can I do something similar with Cmake and Ninja? Is there some equivalent parameter for cmake configuration?

externalNativeBuild {
    cmake {
        arguments "-DANDROID_STL=c++_static"
        abiFilters getAbis()
    }
}

Thanks.

Answer 1

controlling ninja parallelism

Ninja also support the same parameter:

$ ninja --help
usage: ninja [options] [targets...]

[...]

options:
  [...]

  -j N     run N jobs in parallel [default=10, derived from CPUs available]

  [...]

controlling ninja parallelism differentiating compile and link jobs

Now, if you want more granularity. For example, if you would like to limit the number of simultaneous link jobs, or compile jobs, or both.

Starting with CMake 3.11, it is now possible to limit the number of compile and/or link jobs.

You could then configure your project with these options:

-DCMAKE_JOB_POOL_COMPILE:STRING=compile
-DCMAKE_JOB_POOL_LINK:STRING=link
'-DCMAKE_JOB_POOLS:STRING=compile=5;link=2'

Now, if your project end up spawning other child processed that are themselves building projects using ninja, you would have to:

• use the fork of ninja that include Job Server support like it is done in make. Binaries are also available in the associated GitHub releases. See https://github.com/kitware/ninja#readme

• make sure sub-project are also configured with the same -DCMAKE_JOB_ options

in the context of externalNativeBuild

This means you could try something like this:

externalNativeBuild {
    cmake {
        arguments "-DANDROID_STL=c++_static -DCMAKE_JOB_POOL_COMPILE:STRING=compile -DCMAKE_JOB_POOL_LINK:STRING=link '-DCMAKE_JOB_POOLS:STRING=compile=5;link=2'"
        abiFilters getAbis()
    }
}

Answer 2

I was able to fix this by adding:

if (PARALLEL_COMPILE_JOBS)
  set(CMAKE_JOB_POOL_COMPILE compile_job_pool${CMAKE_CURRENT_SOURCE_DIR})
  string (REGEX REPLACE "[^a-zA-Z0-9]+" "_" CMAKE_JOB_POOL_COMPILE ${CMAKE_JOB_POOL_COMPILE})
  set_property(GLOBAL APPEND PROPERTY JOB_POOLS ${CMAKE_JOB_POOL_COMPILE}=${PARALLEL_COMPILE_JOBS})
endif ()
  if (PARALLEL_COMPILE_JOBS)
    message(STATUS "${CMAKE_CURRENT_SOURCE_DIR}: Limiting compiler jobs to ${PARALLEL_COMPILE_JOBS}")
endif ()

to my base CMakeLists.txt, and then in my build.gradle I added to cmake arguments: "-DPARALLEL_COMPILE_JOBS=8" to specify max 8 parallel clang++ compile processes. This works on current Android Studio cmake (3.10) and ninja (1.8.2) versions

Answer 3

I created a workaround: wrap the ninja executable used by Android Studio with a script that calls ninja with all given parameter plus "-j1" at the end.

1. Locate the ninja executable used by Android Studio. Something like, for example, somefolder/Android/Sdk/cmake/3.10.2.4988404/bin/ninja
2. Rename it to something else, like ninja_orig
3. Create a shell script (and add execution permission to it) to replace the original ninja executable at somefolder/Android/Sdk/cmake/3.10.2.4988404/bin/ninja with the following content:

   #!/bin/sh
   somefolder/Android/Sdk/cmake/3.10.2.4988404/bin/ninja_orig $@ -j1
   

Answer 4

Ok it seems it is a bug / missing feature in NDK. I've talked to some "NDK Googlers" and they were not able to help me either. Hopefully it will be supported in later versions of NDK / AS.

Here are the issues you can track the progress:

https://github.com/android-ndk/ndk/issues/983

and

https://issuetracker.google.com/issues/137878831