Building LuxCoreRender: Difference between revisions

From LuxCoreRender Wiki
Jump to navigation Jump to search
← Older edit
 
(268 intermediate revisions by 2 users not shown)
Line 1: Line 1:
''This page is under construction''
This document describes the various processes involved in building LuxCoreRender, starting with version 2.10 “Back-on-track”.
 
LuxCoreRender's build system has been significantly modified for version 2.10. This document therefore renders obsolete all [[Building_LuxCoreRender_(legacy)|previous documents]] related to the compilation of older versions (<=2.9).


This document describes the various processes involved in building LuxCoreRender, starting with version 2.10 “Back-on-track”.


LuxCoreRender's build system has been significantly modified for version 2.10. This document therefore renders obsolete all previous documents related to the compilation of older versions (<=2.9).
<b>TL;DR</b> If you are just interested in a quick, local (but platform limited) compilation of LuxCore C++ samples or Python wheels, you may want to jump directly to [[#LuxCore, Samples & Python Wheels | LuxCore, Samples & Python Wheels]].</br>


For most targets, there are several build approaches, depending on the use case. This document therefore distinguishes '''2 different build workflows'''.




= Audience =
= Audience =


This document is primarily intended for:
This document is intended for:
* LuxCoreRender administrators in charge of releasing the various LuxCoreRender end-products;
* LuxCoreRender administrators in charge of releasing the various LuxCoreRender end-products;
* Developers wishing to contribute to the project;
* Developers wishing to contribute to the project;
* Package maintainers wishing to integrate all or part of LuxCoreRender into a distribution;
* External package maintainers wishing to integrate all or part of LuxCoreRender into a distribution;


This document assumes that the reader is skilled in the following areas:
This document assumes that the reader is skilled in the following areas:
* C/C++ compilation
* C/C++ compilation
* cmake
* [https://cmake.org/ cmake]
* git
* [https://git-scm.com/ git]
* Github
* [https://docs.github.com/en/actions/writing-workflows Github Workflows]
* Python Wheels
* [https://packaging.python.org/en/latest/ Python Packaging], including [https://realpython.com/python-wheels/ Python Wheels format]


Familiarity with [https://conan.io/ Conan] dependency manager may also help.
Familiarity with [https://conan.io/ Conan] dependency manager may also help.


</br>
</br>
This document is not intended for end-users without any knowledge about application building. Such users are invited to look for already compiled binaries.
This document is not intended for end-users without any knowledge about application building. Such users are invited to look for already compiled binaries on the official [https://luxcorerender.org/download/ Download] page.


= Build Targets =
= Build Targets =


== Synoptics ==
== Synoptic ==
LuxCoreRender contains multiple targets, below represented with dependency links:
LuxCoreRender contains several build targets, below represented with dependency links:


{{#mermaid:
{{#mermaid:
Line 36: Line 36:
   init: {
   init: {
     "theme": "forest",
     "theme": "forest",
     "logLevel": "info"
     "logLevel": "info",
    "flowchart": {"curve": "basis"}
   }
   }
}%%
}%%
Line 64: Line 65:
deps(LuxCore Dependencies)
deps(LuxCore Dependencies)
end
end
class Dependencies LUXCOREDEPS;
class Dependencies LUXCOREDEPS;
class deps TARGETDEPS;
class deps TARGETDEPS;


subgraph LuxCore
subgraph LuxCore
lux(LuxCore libraries)
lux(luxcore.so/.dll)
luxa(luxcore.a)
end
end
class LuxCore LUXCORE;
class LuxCore LUXCORE;
class lux TARGETLUXCORE;
class lux TARGETLUXCORE;
class luxa TARGETLUXCORE;


subgraph Python_Wheels[Python Wheels]
subgraph Python_Wheels[Python Wheels]
pylux(pyluxcore.so)
pylux(pyluxcore.so/.dll)
wheels(LuxCore Python Wheels)
wheels(LuxCore Python Wheels)
end
end
Line 89: Line 94:
class luxcoreconsole TARGETSAMPLES;
class luxcoreconsole TARGETSAMPLES;


subgraph PyLuxCoreTools[Python LuxCore Tools]
 
subgraph PyLuxCoreTools
pyluxcon(pyluxcoreconsole)
pyluxcon(pyluxcoreconsole)
pyluxothers(...)
pyluxothers(...)
Line 97: Line 103:
class pyluxothers TARGETNEUTRAL;
class pyluxothers TARGETNEUTRAL;


subgraph Plugins
subgraph Plugins[Python Plugins]
blc(BlendLuxCore)
blc(BlendLuxCore)
otherplugins(...)
otherplugins(...)
Line 106: Line 112:
class Plugins NEUTRAL;
class Plugins NEUTRAL;


 
%% Warning: order matters
deps-->lux
deps-->lux
deps-->pylux
deps-->luxa
deps-->luxcoreui
deps-->luxcoreui
deps-->luxcoreconsole
deps-->luxcoreconsole


lux-->pylux
luxa--->pylux
lux-->luxcoreui
lux-->luxcoreui
lux-->luxcoreconsole
lux-->luxcoreconsole
deps-->pylux




Line 126: Line 133:


Python:::EXTDEP-->pylux
Python:::EXTDEP-->pylux


%% Mermaid version: 8.14.0
%% Mermaid version: 8.14.0
%% https://github.com/mermaid-js/mermaid/blob/8.14.0/src/defaultConfig.js
%% https://github.com/mermaid-js/mermaid/blob/8.14.0/src/defaultConfig.js
 
 
 
}}
}}


Line 136: Line 146:
! Target !! Source Repository !! Content
! Target !! Source Repository !! Content
|-
|-
| <b>External Dependencies</b>|| <code>LuxCoreRender/LuxCoreDeps</code> || A bundled Conan cache populated with LuxCoreRender external dependency binaries, built from sources. Please refer to [https://github.com/LuxCoreRender/LuxCoreDeps/blob/main/README.md LuxCoreDeps README] for more information.
| <b>External Dependencies</b>|| [https://github.com/LuxCoreRender/LuxCoreDeps <code>LuxCoreDeps</code>] || A bundled Conan cache populated with LuxCoreRender external dependency binaries, built from sources. Please refer to [https://github.com/LuxCoreRender/LuxCoreDeps/blob/main/README.md LuxCoreDeps README] for more information.
|-
|-
| <b>LuxCore</b> || <code>LuxCoreRender/LuxCore</code> || LuxCore core binaries, in the form of static and shared libraries: luxcore.so, luxcore_static.lib, luxcore.dll etc.
| <b>LuxCore</b> || [https://github.com/LuxCoreRender/LuxCore <code>LuxCore</code>] || LuxCore core binaries, in the form of static and shared libraries: luxcore.so, luxcore_static.lib, luxcore.dll etc.
|-
|-
| <b>Samples</b>|| <code>LuxCoreRender/LuxCore</code> || Sample C++ programs, illustrating luxcore use, namely <code>luxcoreconsole</code> and <code>luxcoreui</code>
| <b>Samples</b>|| [https://github.com/LuxCoreRender/LuxCore <code>LuxCore</code>] || Sample C++ programs, illustrating luxcore use, namely <code>luxcoreconsole</code> and <code>luxcoreui</code>
|-
|-
| <b>Python Wheels</b>|| <code>LuxCoreRender/LuxCore</code> || Python bindings of core binaries, in the form of Python wheels (1 per pair Platform/Python version).</br> As a byproduct, a Pythonized version of LuxCore shared library is also built (pyluxcore.so).
| <b>Python Wheels</b>|| [https://github.com/LuxCoreRender/LuxCore <code>LuxCore</code>] || Python bindings of core binaries, in the form of Python wheels (1 per pair Platform/Python version).</br> As a byproduct, a Pythonized version of LuxCore shared library is also built (pyluxcore.so).
|-
|-
| <b>Plugins</b>|| <code>LuxCoreRender/<plugin-repo></code> || Plugins to expose LuxCore in various external applications, notably Blender. </br> Mostly written in Python and relying on Python Wheels as runtime dependency.
| <b>Python Plugins</b>|| [https://github.com/LuxCoreRender/BlendLuxCore <code>BlendLuxCore</code>] etc. || Plugins to expose LuxCore in various external applications, notably Blender. </br> Written in Python and relying on Python Wheels as runtime dependency.
|-
|-
| <b>Python LuxCore Tools </br>(PyLuxCoreTools)</b>|| <code>LuxCoreRender/LuxCore</code> || Set of command line tools based on pyluxcore, written in Python.
| <b>PyLuxCoreTools</b>|| [https://github.com/LuxCoreRender/LuxCore <code>LuxCore</code>] || Python LuxCore Tools: set of command line tools based on pyluxcore, written in Python.
|}
|}


Line 157: Line 167:
* MacOS Arm (>=12.0)
* MacOS Arm (>=12.0)


For Python-related targets,LuxCoreRender aims at being available for all Python versions supported at a given time (https://devguide.python.org/versions/).
For Python-related targets, LuxCoreRender aims at being declined for [https://devguide.python.org/versions all Python versions supported at a given time].
 
= Workflow Types =


== Scope of this document ==
We distinguish between two types of build workflows:
* Developer workflows
* Publisher workflows


The scope of this document is:
* External Dependencies
* LuxCore
* Python Wheels
* Samples


Plugin builds are documented in dedicated wiki pages. </br>
''[[#Developer Workflows | Developer workflows]]'' are designed for development, debugging and test steps.
PyLuxCoreTools build documentation is in "TODO" status.


= Build Workflows =
Their purpose is to allow development and verify the code is ready for publication.


== Publisher Workflows ==


=== Introduction ===
''[[#Publisher Workflows | Publisher workflows]]'' are designed to be used by LuxCoreRender administrators to publish a new release of one or more LuxCoreRender components.
These workflows are designed to be used by LuxCoreRender administrators to publish a new release of one or more LuxCoreRender components.


They exclusively take place in a CI/CD Github pipeline.
They exclusively take place in a CI/CD Github pipeline.


These workflows are not designed for debugging or testing the underlying code. Execution of these workflows assumes that the development phase has been correctly completed and the underlying code is ready for final build and release. See [[#Developper Workflows]] to learn how to get code ready for final build.
These workflows are not designed for debugging or testing the underlying code. Execution of these workflows assumes that the development phase has been correctly completed and the underlying code is ready for final build and release. See [[#Developer Workflows | Developer Workflows]] to learn how to get code ready for final build.


Running these workflows require users to be granted of extended rights on LuxCoreRender repositories.
Running these workflows require users to be granted of extended rights on LuxCoreRender repositories.


=== External Dependencies ===
= Developer Workflows =
 
 
== Prerequisites ==
 
<u>Tools</u>
 
You will need several build tools installed and referenced in the PATH.
 
First, ensure you have a suitable toolchain:
 
{| class="wikitable" style="margin:auto"
|-
! OS !! Compiler !! Minimal version
|-
| Windows || MSVC || 194x latest version
|-
| Linux|| gcc || 14
|-
| MacOS Intel|| XCode + llvm + libomp || llvm >= 20 <br> (<code>brew install llvm && brew install libomp</code>)
|-
| MacOS Arm || XCode + llvm + libomp || llvm >= 20 <br> (<code>brew install llvm && brew install libomp</code>)
|}
 
''Toolchain compliance is a very sensitive condition for a successful build. Before filing any claim, please check that your toolchain complies with the above requirements.''
 
 
 
Then ensure the following software is also installed and available in your PATH:
 
* [https://git-scm.com/ Git]
* [https://cli.github.com/ Github CLI] (optional but recommended - for dependency signature checking)
* [https://www.python.org/ Python 3] - any [https://devguide.python.org/versions/ currently supported version]
* [https://conan.io Conan] latest version (pip install conan)
* [https://github.com/pypa/wheel/tree/main Wheel] latest version (pip install wheel)
* [https://cmake.org/ CMake] >= 3.29
* [https://github.com/nektos/act nektos/act] (optional but recommended - for local build based on Github workflows)
 
 
Platform specifics:
* For Windows, ensure the command line is configured for building (vcvarsall.bat).
* For Linux, you'll need <code>pkgconfig</code> in addition.
* For MacOS, you'll need to ensure CC and CXX point to LLVM clang/clang++
 
<u>Accounts</u>
 
A ''Github account'' is mandatory for LuxCore development.
 
Following accounts are also recommended, in order to get in touch with LuxCore community:
* [https://forums.luxcorerender.org/ LuxCoreRender Forums]
* [https://discord.gg/chPGsKV LuxCoreRender Discord channel]
 
== General Developer Workflow ==
 
To compile and verify LuxCore component modifications in development process, there are 3 stages:
* Local plain build: only run build system (cmake) on developer computer
* Local Continuous Integration build: run full Continuous Integration workflow on developer computer
* Server Continuous Integration build: run full Continuous Integration workflow on server
 
<i>Local plain build</i> is generally faster than other builds and does not require to commit changes beforehand. Plain build installs a minimal build configuration (dependencies), but heavily relies on computer build environment for the rest (build tools, system libraries etc.). As a counterpart, it builds only for the platform where it is executed and can be significantly altered by local conditions (build tools local versions, dependency local versions etc.), thus not fully guaranteeing that final publish workflow will succeed.
 
<i>LocalCI </i> is generally faster than Server build, but slower than local plain build. CI workflow installs a complete build environment (compilers, sdk, build tools, containers, dependencies etc.) before running build system. It does not require to commit changes beforehand. However, it builds only for the platform where it is executed, thus not fully guaranteeing that final publish workflow will succeed.
 
<I>Server CI</i>, if successful, should guarantee success when publishing, as it occurs in conditions similar to release workflow (all platforms, complete build environment). As a counterpart, it is generally significantly slower than local builds and also implies to commit and push changes beforehand, even if the code is in unstable state.


The recommended workflow thus strives to make the most of all stages:


''Caveat: if you provide an already-existing release version number during the process, this action will replace the existing release with the newly-built one. This can be what you expected... or not. Be cautious.''
{{#mermaid:  


To build a new release of Dependencies:
%%{ init: {
<!-- External Dependencies -->
  'theme': 'neutral',
  'flowchart': { 'curve': 'monotoneX' }
  }
}%%


{{#mermaid: flowchart TD
flowchart TD;


repo(Go to <a href='https://github.com/LuxCoreRender/LuxCoreDeps'><code>LuxCoreRender/LuxCoreDeps</code></a> Github repository)
fork(Fork Source repository in your own Github space)
action(Open <a href='https://github.com/LuxCoreRender/LuxCoreDeps/actions'><code>Actions</code></a>  menu)
clone("Clone your fork locally</br><code>git clone...</code>")
select(Select <a href='https://github.com/LuxCoreRender/LuxCoreDeps/actions/workflows/release.yml'><code>LuxCore Dependency Releaser</code></a> workflow)
devel(Make modifications in your local fork)
runwf(Click on <code>Run workflow</code> button, in the upper-right corner)
localBuild("<b>Run local plain build</b></br><code>cmake</code>, <code>make</code>...")
choice(Set branch, release version and other options)
localCI("<b>Run local CI</b></br><code>act</code>")
validate(Click <code>Run workflow</code>)
git("Commit, Rebase & Push</br><code>git ...</code>")
wait(Wait for build to complete)
serverCI(<b>Run server CI</b></br><code>Github workflow</code>)
release(Open the <a href='https://github.com/LuxCoreRender/LuxCoreDeps/releases'>LuxCoreDeps release folder</a> in Github, </br> find the newly-built release, which is in Draft mode, </br> and edit it)
PR(PR your changes to Source repository)
complete(Add relevant information: change log, comments on version etc.)
dedraft(Unset Draft mode)
update(Click <code>Publish release</code>/<code>Update release</code> to finalize)


fork-->clone
clone-->devel
devel-->localBuild
localBuild-->localCI
localCI-- "<i>Success</br>(including debug & tests)</i>" -->git
git-->serverCI
serverCI-- <i>Success</i> -->PR


repo:::DEPS-->action
localBuild & localCI & serverCI ----> |<i>Failure</i>| devel
action:::DEPS-->select
select:::DEPS-->runwf
runwf:::DEPS-->choice
choice:::DEPS-->validate
validate:::DEPS-->wait
wait:::WAIT-- if success -->release
release:::DEPS-->complete
complete:::DEPS-->dedraft
dedraft:::DEPS-->update:::DEPS


classDef WAIT fill:#ddd;
classDef BUILD fill:#acc,stroke-width:4px;
classDef DEPS fill:#e4d8ff,stroke:#9488af;
class localBuild BUILD;
class localCI BUILD;
class serverCI BUILD;


}}
}}


=== Python Wheels ===
This is the common process between all components. Each step will then be adapted to each component, as explained below.
 
Comments:
* Running local CI relies on [https://github.com/nektos/act <code>nektos/act</code>]
* Rebase must be made against upstream directory (<code>git pull --rebase upstream <branch></code>)before pushing
* Push must be made towards developer own fork. Caveat: do not try to push directly to upstream
 
== Per-target details ==
 
=== External Dependencies ===
 
==== Local Plain Build ====
<i>This feature is not available yet. Meanwhile, you may use Local CI instead.</i>
 
==== Local Continuous Integration ====
 
Run <code>act</code> on workflows located in <code>.github/workflows</code> folder, for instance <code>.github/workflows/build.yml</code> workflow.</br>
 
For more convenience, a wrapper script is also provided in <code>LuxCoreDeps</code> repository: <code>utils/debug.sh</code> (Linux only, at the moment).


==== Server Continuous Integration ====


==== Standard Build ====
This process relies on the following Github workflow: [https://github.com/LuxCoreRender/LuxCoreDeps/actions/workflows/checker.yml <code>LuxCore Dependency Checker</code>].
<!-- Python Wheels -->
</br>Important points:
* Your changes must have been committed and pushed to repository before.
* The Checker workflow is actually also triggered on push event, so you may not have to run it by yourself


''Caveat: if you provide an already-existing release version number during the process, this action will replace the existing release with the newly-built one. This can be what you expected... or not. Be cautious.''
If build succeeds, you will find the expected outputs in Artifacts section of the action run.


Prerequisite: you must have release a compatible version of External Dependencies (LuxCoreDeps) before building Python Wheels.
=== LuxCore, Samples & Python Wheels ===


To build a new release of Python Wheels:
==== Local Plain Build ====


{{#mermaid: flowchart TD
This method internally relies on <code>cmake</code> but, for more convenience, everything has been wrapped into <code>make</code>-like statements.


repo(Go to <a href='https://github.com/LuxCoreRender/LuxCore'><code>LuxCoreRender/LuxCoreDeps</code></a> Github repository)
First, we assume you have correctly completed the [[#Prerequisites_2 | Prerequisites]].  
action(Open <a href='https://github.com/LuxCoreRender/LuxCore/actions'><code>Actions</code></a>  menu)
 
select(Select <a href='https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-releaser.yml'><code>LuxCore Wheels Releaser</code></a> workflow)
Second, we also assume you've already git-cloned LuxCore on your local machine, opened a terminal, changed directory to LuxCore folder and git-checked out the branch that you want to build (+ for Windows: initialized build environment with <code>vcvarsall.bat</code>)
runwf(Click on <code>Run workflow</code> drop-down button, in the upper-right corner)
choice(Set branch, release version and other options)
validate(Click <code>Run workflow</code>)
wait(Wait for build to complete)
release(Open the <a href='https://github.com/LuxCoreRender/LuxCore/releases'>LuxCore release folder</a> in Github, </br> find the newly-built release, which is in Draft mode, </br> and edit it)
complete(Add relevant information: change log, comments on version etc.)
dedraft(Unset Draft mode)
update(Click <code>Publish release</code>/<code>Update release</code> to finalize)


repo-->action
From here on, just enter the magic words:
action-->select
<pre>
select-->runwf
make deps
runwf-->choice
make
choice-->validate
</pre>
validate-->wait
wait:::WAIT-- if success -->release
release-->complete
complete-->dedraft
dedraft-->update


classDef WAIT fill:#ddd;
This should retrieve and install LuxCore dependencies, configure the project and run the build. That's all!


}}


==== Test & Debug Build ====
For subsequent builds, if no modification have been made to dependencies, you'll just have to re-enter: <pre>make</pre>


Standard build may not be suitable for testing and debugging needs (for instance, when modifying build process...). In such situations, there are 3 options:
* Server build: run workflow on server, but without creating any release
* Local build - run workflow: run workflow on local computer (user PC)
* Local build - developper:


<u>Server build</u>
If you want to target a specific component, you can also use the following verbs:
* <code>make luxcore</code>
* <code>make pyluxcore</code>
* <code>make luxcoreui</code>
* <code>make luxcoreconsole</code>


The process is nearly the same as canonical build, but the release creation steps are skipped. For that purpose, another Github workflow is executed: ''LuxCore Dependency Builder''.</br>
Build type can be selected by setting environment variable <code>LUX_BUILD_TYPE</code>. Valid values are 'Release' (default) and 'Debug'.
Please note that your changes must have been committed to repository beforehand.


<!-- Debug -->
{{#mermaid: flowchart TD


repo(Go to <a href='https://github.com/LuxCoreRender/LuxCore'><code>LuxCoreRender/LuxCoreDeps</code></a> Github repository)
You can also build Doxygen documentation with the following statement:
action(Open <a href='https://github.com/LuxCoreRender/LuxCore/actions'><code>Actions</code></a>  menu)
* <code>make doc</code>
select(Select <a href='https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-builder.yml'><code>LuxCore Python Wheels Builder</code></a> workflow)
runwf(Click on <code>Run workflow</code> drop-down button, in the upper-right corner)
choice(Select Branch and Build type)
validate(Click on <code>Run workflow</code> button)
wait(Wait for build to complete)




And you can remove build outputs with:
* <code>make clean</code>
* <code>make clear</code>
The latter will remove everything and will require to reinstall dependencies (<code>make deps</code>).


repo-->action
action-->select
select-->runwf
runwf-->choice
choice-->validate
validate-->wait:::WAIT


classDef WAIT fill:#ddd;
Starting from version 2.11, this method can build a "test wheel". This is a simplified wheel, only for current platform and current Python version, for testing purpose. The verb is:
* <code>make wheel-test</code>


}}


If build succeeds, you will find the expected outputs in Artifacts section of the action run.
By default, this method builds a release version, striped from all symbols. If you want to build a debug version, enter <code>export LUX_BUILD_TYPE=Debug</code> before any <code>make ...</code> command.


<u>Local build</u></br>
==== Local Continuous Integration ====
Thanks to [https://github.com/nektos/act nektos/act], it is possible Github Workflows locally.


After installation (left to the reader), <code>act</code> can be called directly on command line. For more convenience, a script is also provided in <code>LuxCoreDeps</code> repository: <code>utils/debug.sh</code> (Linux only, at the moment).
This method is complementary to Local plain build, with the following additional capabilities:
* test the LuxCore build in an environment close to the release environment (manylinux container, for instance)
* can build wheels
* debug any changes made to the build workflow


==== Wheels Deployment ====
It consists in running Github workflows locally, via <code>act</code>. Github workflows are located in <code>.github/workflows</code> folder.


As far as wheels are concerned, in addition to building them, it is also necessary to upload them to PyPi ("deploy" them), so that they are available for pip installation.
<code>act</code> invokation can be made on command line but, for convenience, a wrapper script is provided: <code>build-helpers/debug/debug_wheels.sh</code>.


Prerequisite: you must have released a set of Python Wheels beforehand.
<i>Caveat: this build can produce wheels, but only for the host platform (no cross-compiling).</i>


{{#mermaid: flowchart TD
==== Server Continuous Integration ====


repo(Go to <a href='https://github.com/LuxCoreRender/LuxCore'><code>LuxCoreRender/LuxCoreDeps</code></a> Github repository)
The process is nearly the same as publisher build, but the release steps are skipped. For that purpose, another Github workflow has to be executed: [https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-builder.yml <code>LuxCore Python Wheels Builder</code>].
action(Open <a href='https://github.com/LuxCoreRender/LuxCore/actions'><code>Actions</code></a>  menu)
select(Select <a href='https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-publisher.yml'><code>LuxCore Python Wheels Publisher</code></a> workflow)
runwf(Click on <code>Run workflow</code> drop-down button, in the upper-right corner)
choice(Select Branch, set the Tag to publish and uncheck Publish on Pypi Test if needed)
validate(Click on <code>Run workflow</code> button)
wait(Wait for deployment to complete)


Please note that your changes must have been committed to repository before running the workflow.


If build succeeds, you will find the expected outputs in Artifacts section of the action run.


repo-->action
= Publisher Workflows =
action-->select
select-->runwf
runwf-->choice
choice-->validate
validate-->wait:::WAIT


classDef WAIT fill:#ddd;
== Prerequisites ==


}}
For these workflows, a Github account is required, with extended rights on the source repositories.


=== Samples ===
== General Workflow ==


As we plan to replace Samples by Python tools, there is no Releaser workflow, just a Builder one. Releases have to be created manually, on the basis of Builder's artifacts.
The general workflow for publishing is in 3 parts:


{{#mermaid: flowchart TD
{{#mermaid:  


repo(Go to <a href='https://github.com/LuxCoreRender/LuxCore'><code>LuxCoreRender/LuxCore</code></a> Github repository)
%%{ init: {
action(Open <a href='https://github.com/LuxCoreRender/LuxCore/actions'><code>Actions</code></a>  menu)
  'theme': 'neutral',
select(Select <a href='https://github.com/LuxCoreRender/LuxCore/actions/workflows/sample-builder.yml'><code>LuxCore Samples Builder</code></a> workflow)
  'flowchart': { 'curve': 'monotoneX' }
  }
}%%
flowchart TD;
subgraph Build["<center><b>BUILD & RELEASE DRAFT </br>(Run Releaser workflow)</b></center>"]
padBuild(" "):::INVISIBLE
repo("Go to <code>[Source repository]</code> on Github and open <code>Actions menu</code>")
select("Select <code>[Releaser workflow]</code>")
runwf(Click on <code>Run workflow</code> button, in the upper-right corner)
runwf(Click on <code>Run workflow</code> button, in the upper-right corner)
choice(Set branch)
branch(Select <code>Branch</code> in drop-down menu)
options(Set workflow options)
validate(Click <code>Run workflow</code>)
validate(Click <code>Run workflow</code>)
wait(Wait for build to complete)
wait(Wait for build to complete)
end
subgraph Release["<center><b>MAKE FINAL RELEASE</br>(Switch release to final status)</b></center>"]
padRelease(" "):::INVISIBLE
release("Open <code>[Release folder]</code>,</br> find the newly-built release (in Draft mode) </br> and edit it")
complete(Add relevant information: change log, comments on version etc.)
dedraft(Unset <code>Draft</code> checkbox)
update(Click <code>Publish release</code>/<code>Update release</code> to finalize)
end
subgraph Deploy["<center><b>PUBLISH</br>(Run Publisher workflow)</b></center>"]
padDeploy(" ")
repoDeploy("Go to <code>[Source repository]</code> on Github and open <code>[Actions menu]</code>")
selectDeploy("Select <code>[Publisher workflow]</code>")
runwfDeploy("Click on <code>Run workflow</code> drop-down button, in the upper-right corner")
choiceDeploy("Select <code>Branch</code>, set the <code>Tag to publish</code> and uncheck <code>Publish on Pypi Test</code> if needed")
validateDeploy("Click on <code>Run workflow</code> button")
waitDeploy("Wait for deployment to complete")
end
padBuild---repo
repo-->select
select-->runwf
runwf-->branch
branch-->options
options-->validate
validate-->wait:::WAIT
linkStyle 0 stroke:#fff,stroke-width:0px,color:blue;
padRelease---release
release-->complete
complete-->dedraft
dedraft-->update
linkStyle 7 stroke:#fff,stroke-width:0px,color:blue;


padDeploy:::INVISIBLE---repoDeploy
repoDeploy-->selectDeploy
selectDeploy-->runwfDeploy
runwfDeploy-->choiceDeploy
choiceDeploy-->validateDeploy
validateDeploy-->waitDeploy:::WAIT
linkStyle 11 stroke:#fff,color:blue,stroke-width:0px;




repo:::TARGETSAMPLES-->action
Build-- <i>if success</i> -->Release
action:::TARGETSAMPLES-->select
Release-- <i>if success & if needed</i> -->Deploy
select:::TARGETSAMPLES-->runwf
runwf:::TARGETSAMPLES-->choice
choice:::TARGETSAMPLES-->validate
validate:::TARGETSAMPLES-->wait:::WAIT


classDef WAIT fill:#ddd;
classDef WAIT fill:#ddd;
classDef SAMPLES fill:#ffffdd,font-weight:bold,font-size:1.1rem,stroke:#999999;
classDef DEPS fill:#e4d8ff,stroke:#9488af;
classDef TARGETSAMPLES fill:#eeeecc,font-weight:bold,font-size:0.9rem,stroke:#999999;
classDef INVISIBLE fill:#ffffffff,stroke:#ffffffff,stroke-width:0px,font-color:#ffffffff;
classDef OPTIONAL stroke-dasharray: 5 5;
 
class Deploy OPTIONAL;


%% Note: padXXX is a poor hack to create a top-margin in subgraph...
}}
}}


== Developper Workflows ==
A few comments:
* <code>[bracketed tokens]</code> refer to data specific to each component, which will be detailed below.
* Publish part is optional, it will depends on the target
* ''Caveat: if you provide an already-existing release version number during the process, the workflow will replace the existing release with the newly-built one. This can be what you expected... or not. Be cautious.''


=== Introduction ===
== Per-target details ==


These workflows are designed for development, debugging and test steps.
=== External Dependencies ===


Their purpose is to prepare the code for publication. To fully understand this section, we recommend you take a look at the [[#Publisher Workflows]] beforehand.
* Source repository: [https://github.com/LuxCoreRender/LuxCoreDeps <code>LuxCoreRender/LuxCoreDeps</code>]
* Releaser workflow: [https://github.com/LuxCoreRender/LuxCoreDeps/actions/workflows/release.yml <code>LuxCore Dependency Releaser</code>]
* Release folder: [https://github.com/LuxCoreRender/LuxCoreDeps/releases <code>Releases</code>]
* Publisher workflow: ∅


=== External Dependencies ===
As a byproduct, External Dependencies are not intended to be deployed, so no Publisher workflow is provided.


To build external dependencies in development process, there are 2 options:
=== Python Wheels ===
* Server build: run workflow on server, but without creating any release
* Local build: run workflow on local computer (user PC)


Local build is generally faster than server build and does not require to commit changes beforehand. However, it builds only for the platform where it is executed and can be slightly altered by local conditions (build tools local versions etc.).


Therefore, the recommended workflow is:
* Fork LuxCoreDeps in Github and clone your fork locally
* Make development/debugging cycles in local environment with the help of local build
* When local build is successful, commit changes, push to your Github fork and run server build (actually, the run should be triggered by push action). Check that the build succeeds for all platforms, in server conditions. If not, fix the issues and push again, till everything is ok.
* When server build succeeds, PR to LuxCore/LuxCoreDeps


==== Server build ====
As far as wheels are concerned, in addition to building them, it is also necessary to upload them to [https://pypi.org/ PyPi] ("publish" them), so that they are available for [https://pypi.org/project/pip pip] installation.


The process is nearly the same as publisher build, but the release creation steps are skipped. For that purpose, another Github workflow is executed: ''LuxCore Dependency Checker''.</br>
* Source repository: [https://github.com/LuxCoreRender/LuxCore <code>LuxCoreRender/LuxCore</code>]
Please note that your changes must have been committed to repository beforehand.
* Releaser workflow: [https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-releaser.yml <code>LuxCore Wheels Releaser</code>]
* Release folder: [https://github.com/LuxCoreRender/LuxCore/releases <code>Releases</code>]
* Publisher workflow: [https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-publisher.yml <code>LuxCore Wheels Publisher</code>]


{{#mermaid: flowchart TD
''Prerequisite: you must have released a compatible version of External Dependencies (LuxCoreDeps) before building Python Wheels.''


repo(Go to <a href='https://github.com/LuxCoreRender/LuxCoreDeps'><code>LuxCoreRender/LuxCoreDeps</code></a> Github repository)
=== Samples ===
action(Open <a href='https://github.com/LuxCoreRender/LuxCoreDeps/actions'><code>Actions</code></a>  menu)
 
select(Select <a href='https://github.com/LuxCoreRender/LuxCoreDeps/actions/workflows/check.yml'><code>LuxCore Dependency Checker</code></a> workflow)
As we plan to replace Samples by equivalent Python tools and thus deprecate them, the workflow has been limited to the Release part (no publish).
runwf(Click on <code>Run workflow</code> button, in the upper-right corner)
wait(Wait for build to complete)


* Source repository: [https://github.com/LuxCoreRender/LuxCore <code>LuxCoreRender/LuxCore</code>]
* Releaser workflow: [https://github.com/LuxCoreRender/LuxCore/actions/workflows/sample-releaser.yml <code>LuxCore Samples Releaser</code>]
* Release folder: [https://github.com/LuxCoreRender/LuxCore/releases <code>Releases</code>]
* Publisher workflow: ∅


=== BlendLuxCore ===


repo:::DEPS-->action
<i>TODO</i>
action:::DEPS-->select
select:::DEPS-->runwf
runwf:::DEPS-->wait:::WAIT


classDef DEPS fill:#e4d8ff,stroke:#9488af;
=== PyLuxCoreTools ===
classDef WAIT fill:#ddd;


}}
<i>TODO</i>


If build succeeds, you will find the expected outputs in Artifacts section of the action run.
= Version Management =


==== Local build ====
== Semantic Versioning ==
This build relies on [https://github.com/nektos/act nektos/act]. Thanks to this very smart tool, it is possible Github Workflows locally. See install instructions on their website.


After installation, <code>act</code> can be executed directly on command line: call it to run workflows located in <code>.github/workflows</code> folder.</br>
Starting for LuxCore version 2.10.0, we require that LuxCoreRender applies [https://semver.org/ Semantic Versioning]. Please strictly enforce this rule when incrementing version numbers.
For more convenience, a wrapper script is also provided in <code>LuxCoreDeps</code> repository: <code>utils/debug.sh</code> (Linux only, at the moment).


=== LuxCore & Samples ===
Remark: component versions does not have to be synced. For instance, in strict application of the standard, the newly created LuxCoreDeps will start at 1.0.0, while at the same time LuxCore is already at 2.10.0


To build LuxCore & Samples in development process, there are 3 methods:
== Dependency Version pointer ==
* Local build - make wrapper
* Local build - Github workflows
* Server build: run workflow on server, but without creating any release
</br>
{| class="wikitable" style="margin:auto"
|-
! Method !! Description !! Pros !! Cons
|-
| '''Local build - make wrapper''' || Execute cmake statements via make wrapper || Very fast (incremental). Do not require to commit changes beforehand || Affected by local conditions (tool versions, libc version, system deps...). Build only for local platform
|-
| '''Local build - Github workflows''' || Execute Github workflow locally || Fast. More robust to local conditions (containerized). Do not require to commit changes beforehand || Slower than make wrapper (rebuild all every time). Can be slightly affected by local conditions
|-
| '''Server build''' || Execute Github workfow on server, without issuing a release || Build in Release environment || Slow. Requires to commit changes beforehand.
|}


== Package Maintainers ==
To make LuxCore know which version of dependencies it must be built against, we use a pointer.


= Miscellaneous =
This pointer is stored in the file <code>build-helpers/build-settings.json</code>, under the path <code>Dependencies/release</code>.


TODO
Make sure you keep it up-to-date when you upgrade LuxCore components.
Python Wheels: Pointer to deps
Semantic Versioning

Latest revision as of 18:13, 10 August 2025

This document describes the various processes involved in building LuxCoreRender, starting with version 2.10 “Back-on-track”.

LuxCoreRender's build system has been significantly modified for version 2.10. This document therefore renders obsolete all previous documents related to the compilation of older versions (<=2.9).


TL;DR If you are just interested in a quick, local (but platform limited) compilation of LuxCore C++ samples or Python wheels, you may want to jump directly to LuxCore, Samples & Python Wheels.


Audience

This document is intended for:

  • LuxCoreRender administrators in charge of releasing the various LuxCoreRender end-products;
  • Developers wishing to contribute to the project;
  • External package maintainers wishing to integrate all or part of LuxCoreRender into a distribution;

This document assumes that the reader is skilled in the following areas:

Familiarity with Conan dependency manager may also help.


This document is not intended for end-users without any knowledge about application building. Such users are invited to look for already compiled binaries on the official Download page.

Build Targets

Synoptic

LuxCoreRender contains several build targets, below represented with dependency links:

Target Source Repository Content
External Dependencies LuxCoreDeps A bundled Conan cache populated with LuxCoreRender external dependency binaries, built from sources. Please refer to LuxCoreDeps README for more information.
LuxCore LuxCore LuxCore core binaries, in the form of static and shared libraries: luxcore.so, luxcore_static.lib, luxcore.dll etc.
Samples LuxCore Sample C++ programs, illustrating luxcore use, namely luxcoreconsole and luxcoreui
Python Wheels LuxCore Python bindings of core binaries, in the form of Python wheels (1 per pair Platform/Python version).
As a byproduct, a Pythonized version of LuxCore shared library is also built (pyluxcore.so).
Python Plugins BlendLuxCore etc. Plugins to expose LuxCore in various external applications, notably Blender.
Written in Python and relying on Python Wheels as runtime dependency.
PyLuxCoreTools LuxCore Python LuxCore Tools: set of command line tools based on pyluxcore, written in Python.

Targeted platforms

LuxCoreRender aims at being available on the following 4 platforms:

  • Linux (glibc 2.28+)
  • Windows
  • MacOS Intel (>=10.15)
  • MacOS Arm (>=12.0)

For Python-related targets, LuxCoreRender aims at being declined for all Python versions supported at a given time.

Workflow Types

We distinguish between two types of build workflows:

  • Developer workflows
  • Publisher workflows


Developer workflows are designed for development, debugging and test steps.

Their purpose is to allow development and verify the code is ready for publication.


Publisher workflows are designed to be used by LuxCoreRender administrators to publish a new release of one or more LuxCoreRender components.

They exclusively take place in a CI/CD Github pipeline.

These workflows are not designed for debugging or testing the underlying code. Execution of these workflows assumes that the development phase has been correctly completed and the underlying code is ready for final build and release. See Developer Workflows to learn how to get code ready for final build.

Running these workflows require users to be granted of extended rights on LuxCoreRender repositories.

Developer Workflows

Prerequisites

Tools

You will need several build tools installed and referenced in the PATH.

First, ensure you have a suitable toolchain:

OS Compiler Minimal version
Windows MSVC 194x latest version
Linux gcc 14
MacOS Intel XCode + llvm + libomp llvm >= 20
(brew install llvm && brew install libomp)
MacOS Arm XCode + llvm + libomp llvm >= 20
(brew install llvm && brew install libomp)

Toolchain compliance is a very sensitive condition for a successful build. Before filing any claim, please check that your toolchain complies with the above requirements.


Then ensure the following software is also installed and available in your PATH:


Platform specifics:

  • For Windows, ensure the command line is configured for building (vcvarsall.bat).
  • For Linux, you'll need pkgconfig in addition.
  • For MacOS, you'll need to ensure CC and CXX point to LLVM clang/clang++

Accounts

A Github account is mandatory for LuxCore development.

Following accounts are also recommended, in order to get in touch with LuxCore community:

General Developer Workflow

To compile and verify LuxCore component modifications in development process, there are 3 stages:

  • Local plain build: only run build system (cmake) on developer computer
  • Local Continuous Integration build: run full Continuous Integration workflow on developer computer
  • Server Continuous Integration build: run full Continuous Integration workflow on server

Local plain build is generally faster than other builds and does not require to commit changes beforehand. Plain build installs a minimal build configuration (dependencies), but heavily relies on computer build environment for the rest (build tools, system libraries etc.). As a counterpart, it builds only for the platform where it is executed and can be significantly altered by local conditions (build tools local versions, dependency local versions etc.), thus not fully guaranteeing that final publish workflow will succeed.

LocalCI is generally faster than Server build, but slower than local plain build. CI workflow installs a complete build environment (compilers, sdk, build tools, containers, dependencies etc.) before running build system. It does not require to commit changes beforehand. However, it builds only for the platform where it is executed, thus not fully guaranteeing that final publish workflow will succeed.

Server CI, if successful, should guarantee success when publishing, as it occurs in conditions similar to release workflow (all platforms, complete build environment). As a counterpart, it is generally significantly slower than local builds and also implies to commit and push changes beforehand, even if the code is in unstable state.

The recommended workflow thus strives to make the most of all stages:

This is the common process between all components. Each step will then be adapted to each component, as explained below.

Comments:

  • Running local CI relies on nektos/act
  • Rebase must be made against upstream directory (git pull --rebase upstream <branch>)before pushing
  • Push must be made towards developer own fork. Caveat: do not try to push directly to upstream

Per-target details

External Dependencies

Local Plain Build

This feature is not available yet. Meanwhile, you may use Local CI instead.

Local Continuous Integration

Run act on workflows located in .github/workflows folder, for instance .github/workflows/build.yml workflow.

For more convenience, a wrapper script is also provided in LuxCoreDeps repository: utils/debug.sh (Linux only, at the moment).

Server Continuous Integration

This process relies on the following Github workflow: LuxCore Dependency Checker.
Important points:

  • Your changes must have been committed and pushed to repository before.
  • The Checker workflow is actually also triggered on push event, so you may not have to run it by yourself

If build succeeds, you will find the expected outputs in Artifacts section of the action run.

LuxCore, Samples & Python Wheels

Local Plain Build

This method internally relies on cmake but, for more convenience, everything has been wrapped into make-like statements.

First, we assume you have correctly completed the Prerequisites.

Second, we also assume you've already git-cloned LuxCore on your local machine, opened a terminal, changed directory to LuxCore folder and git-checked out the branch that you want to build (+ for Windows: initialized build environment with vcvarsall.bat)

From here on, just enter the magic words: 

make deps
make

This should retrieve and install LuxCore dependencies, configure the project and run the build. That's all!


For subsequent builds, if no modification have been made to dependencies, you'll just have to re-enter: 

make


If you want to target a specific component, you can also use the following verbs:

  • make luxcore
  • make pyluxcore
  • make luxcoreui
  • make luxcoreconsole

Build type can be selected by setting environment variable LUX_BUILD_TYPE. Valid values are 'Release' (default) and 'Debug'.


You can also build Doxygen documentation with the following statement:

  • make doc


And you can remove build outputs with:

  • make clean
  • make clear

The latter will remove everything and will require to reinstall dependencies (make deps).


Starting from version 2.11, this method can build a "test wheel". This is a simplified wheel, only for current platform and current Python version, for testing purpose. The verb is:

  • make wheel-test


By default, this method builds a release version, striped from all symbols. If you want to build a debug version, enter export LUX_BUILD_TYPE=Debug before any make ... command.

Local Continuous Integration

This method is complementary to Local plain build, with the following additional capabilities:

  • test the LuxCore build in an environment close to the release environment (manylinux container, for instance)
  • can build wheels
  • debug any changes made to the build workflow

It consists in running Github workflows locally, via act. Github workflows are located in .github/workflows folder.

act invokation can be made on command line but, for convenience, a wrapper script is provided: build-helpers/debug/debug_wheels.sh.

Caveat: this build can produce wheels, but only for the host platform (no cross-compiling).

Server Continuous Integration

The process is nearly the same as publisher build, but the release steps are skipped. For that purpose, another Github workflow has to be executed: LuxCore Python Wheels Builder.

Please note that your changes must have been committed to repository before running the workflow.

If build succeeds, you will find the expected outputs in Artifacts section of the action run.

Publisher Workflows

Prerequisites

For these workflows, a Github account is required, with extended rights on the source repositories.

General Workflow

The general workflow for publishing is in 3 parts:

A few comments:

  • [bracketed tokens] refer to data specific to each component, which will be detailed below.
  • Publish part is optional, it will depends on the target
  • Caveat: if you provide an already-existing release version number during the process, the workflow will replace the existing release with the newly-built one. This can be what you expected... or not. Be cautious.

Per-target details

External Dependencies

As a byproduct, External Dependencies are not intended to be deployed, so no Publisher workflow is provided.

Python Wheels

As far as wheels are concerned, in addition to building them, it is also necessary to upload them to PyPi ("publish" them), so that they are available for pip installation.

Prerequisite: you must have released a compatible version of External Dependencies (LuxCoreDeps) before building Python Wheels.

Samples

As we plan to replace Samples by equivalent Python tools and thus deprecate them, the workflow has been limited to the Release part (no publish).

BlendLuxCore

TODO

PyLuxCoreTools

TODO

Version Management

Semantic Versioning

Starting for LuxCore version 2.10.0, we require that LuxCoreRender applies Semantic Versioning. Please strictly enforce this rule when incrementing version numbers.

Remark: component versions does not have to be synced. For instance, in strict application of the standard, the newly created LuxCoreDeps will start at 1.0.0, while at the same time LuxCore is already at 2.10.0

Dependency Version pointer

To make LuxCore know which version of dependencies it must be built against, we use a pointer.

This pointer is stored in the file build-helpers/build-settings.json, under the path Dependencies/release.

Make sure you keep it up-to-date when you upgrade LuxCore components.

Retrieved from "https://wiki.luxcorerender.org/index.php?title=Building_LuxCoreRender&oldid=2392"