Building LuxCoreRender: Difference between revisions

From LuxCoreRender Wiki
Jump to navigation Jump to search
 
(190 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This document describes the various processes involved in building LuxCoreRender, starting with version 2.10 “Back-on-track”.
This document describes the various processes involved in building LuxCoreRender, starting with version 2.10 “Back-on-track”.


LuxCoreRender's build system has been significantly modified for version 2.10. This document therefore renders obsolete all previous [[Building_LuxCoreRender_(legacy)|documents]] related to the compilation of older versions (<=2.9).
LuxCoreRender's build system has been significantly modified for version 2.10. This document therefore renders obsolete all [[Building_LuxCoreRender_(legacy)|previous documents]] related to the compilation of older versions (<=2.9).
 
 
<b>TL;DR</b> If you are just interested in a quick, local (but platform limited) compilation of LuxCore C++ samples or Python wheels, you may want to jump directly to [[#LuxCore, Samples & Python Wheels | LuxCore, Samples & Python Wheels]].</br>
 




Line 21: Line 25:


</br>
</br>
This document is not intended for end-users without any knowledge about application building. Such users are invited to look for already compiled binaries.
This document is not intended for end-users without any knowledge about application building. Such users are invited to look for already compiled binaries on the official [https://luxcorerender.org/download/ Download] page.


= Build Targets =
= Build Targets =


== Synoptics ==
== Synoptic ==
LuxCoreRender contains multiple targets, below represented with dependency links:
LuxCoreRender contains several build targets, below represented with dependency links:


{{#mermaid:
{{#mermaid:
Line 67: Line 71:


subgraph LuxCore
subgraph LuxCore
lux(LuxCore libraries)
lux(luxcore.so/.dll)
luxa(luxcore.a)
end
end
class LuxCore LUXCORE;
class LuxCore LUXCORE;
class lux TARGETLUXCORE;
class lux TARGETLUXCORE;
class luxa TARGETLUXCORE;


subgraph Python_Wheels[Python Wheels]
subgraph Python_Wheels[Python Wheels]
pylux(pyluxcore.so)
pylux(pyluxcore.so/.dll)
wheels(LuxCore Python Wheels)
wheels(LuxCore Python Wheels)
end
end
Line 108: Line 114:
%% Warning: order matters
%% Warning: order matters
deps-->lux
deps-->lux
deps-->luxa
deps-->luxcoreui
deps-->luxcoreui
deps-->luxcoreconsole
deps-->luxcoreconsole


lux--->pylux
luxa--->pylux
lux-->luxcoreui
lux-->luxcoreui
lux-->luxcoreconsole
lux-->luxcoreconsole
Line 162: Line 169:
For Python-related targets, LuxCoreRender aims at being declined for [https://devguide.python.org/versions all Python versions supported at a given time].
For Python-related targets, LuxCoreRender aims at being declined for [https://devguide.python.org/versions all Python versions supported at a given time].


= Build Workflows =
= Workflow Types =


== Introduction ==
We distinguish between two types of build workflows:
 
* Developer workflows
We'll distinguish between two types of workflows:
* Publisher workflows
* Publisher workflows
* Developer workflows




''Publisher workflows'' are designed to be used by LuxCoreRender administrators to publish a new release of one or more LuxCoreRender components.
''[[#Developer Workflows | Developer workflows]]'' are designed for development, debugging and test steps.


They exclusively take place in a CI/CD Github pipeline.
Their purpose is to allow development and verify the code is ready for publication.


These workflows are not designed for debugging or testing the underlying code. Execution of these workflows assumes that the development phase has been correctly completed and the underlying code is ready for final build and release. See [[#Developper Workflows]] to learn how to get code ready for final build.


Running these workflows require users to be granted of extended rights on LuxCoreRender repositories.
''[[#Publisher Workflows | Publisher workflows]]'' are designed to be used by LuxCoreRender administrators to publish a new release of one or more LuxCoreRender components.


They exclusively take place in a CI/CD Github pipeline.


''Developer workflows'' are designed for development, debugging and test steps.
These workflows are not designed for debugging or testing the underlying code. Execution of these workflows assumes that the development phase has been correctly completed and the underlying code is ready for final build and release. See [[#Developer Workflows | Developer Workflows]] to learn how to get code ready for final build.


Their purpose is to prepare the code for publication. Thus, to fully understand this section, we recommend you to have a look at the [[#Publisher Workflows]] beforehand.
Running these workflows require users to be granted of extended rights on LuxCoreRender repositories.


== Publisher Workflows ==
= Developer Workflows =




=== External Dependencies ===
== Prerequisites ==


<u>Tools</u>


''Caveat: if you provide an already-existing release version number during the process, this action will replace the existing release with the newly-built one. This can be what you expected... or not. Be cautious.''
You will need several build tools installed and referenced in the PATH.


To build a new release of Dependencies, follow the process below:
First, ensure you have a suitable toolchain:
<!-- External Dependencies -->


{{#mermaid: flowchart TD
{| class="wikitable" style="margin:auto"
|-
! OS !! Compiler !! Minimal version
|-
| Windows || MSVC || 194x latest version
|-
| Linux|| gcc || 14
|-
| MacOS Intel|| XCode + llvm + libomp || llvm >= 20 <br> (<code>brew install llvm && brew install libomp</code>)
|-
| MacOS Arm || XCode + llvm + libomp || llvm >= 20 <br> (<code>brew install llvm && brew install libomp</code>)
|}


repo(Go to <a href='https://github.com/LuxCoreRender/LuxCoreDeps'><code>LuxCoreRender/LuxCoreDeps</code></a> Github repository)
''Toolchain compliance is a very sensitive condition for a successful build. Before filing any claim, please check that your toolchain complies with the above requirements.''
action(Open <a href='https://github.com/LuxCoreRender/LuxCoreDeps/actions'><code>Actions</code></a>  menu)
select(Select <a href='https://github.com/LuxCoreRender/LuxCoreDeps/actions/workflows/release.yml'><code>LuxCore Dependency Releaser</code></a> workflow)
runwf(Click on <code>Run workflow</code> button, in the upper-right corner)
choice(Set branch, release version and other options)
validate(Click <code>Run workflow</code>)
wait(Wait for build to complete)
release(Open the <a href='https://github.com/LuxCoreRender/LuxCoreDeps/releases'>LuxCoreDeps release folder</a> in Github, </br> find the newly-built release, which is in Draft mode, </br> and edit it)
complete(Add relevant information: change log, comments on version etc.)
dedraft(Unset Draft mode)
update(Click <code>Publish release</code>/<code>Update release</code> to finalize)




repo:::DEPS-->action
action:::DEPS-->select
select:::DEPS-->runwf
runwf:::DEPS-->choice
choice:::DEPS-->validate
validate:::DEPS-->wait
wait:::WAIT-- if success -->release
release:::DEPS-->complete
complete:::DEPS-->dedraft
dedraft:::DEPS-->update:::DEPS


classDef WAIT fill:#ddd;
Then ensure the following software is also installed and available in your PATH:
classDef DEPS fill:#e4d8ff,stroke:#9488af;


}}
* [https://git-scm.com/ Git]
* [https://cli.github.com/ Github CLI] (optional but recommended - for dependency signature checking)
* [https://www.python.org/ Python 3] - any [https://devguide.python.org/versions/ currently supported version]
* [https://conan.io Conan] latest version (pip install conan)
* [https://github.com/pypa/wheel/tree/main Wheel] latest version (pip install wheel)
* [https://cmake.org/ CMake] >= 3.29
* [https://github.com/nektos/act nektos/act] (optional but recommended - for local build based on Github workflows)


=== Python Wheels ===


Platform specifics:
* For Windows, ensure the command line is configured for building (vcvarsall.bat).
* For Linux, you'll need <code>pkgconfig</code> in addition.
* For MacOS, you'll need to ensure CC and CXX point to LLVM clang/clang++


==== Build ====
<u>Accounts</u>
<!-- Python Wheels -->


''Caveat: if you provide an already-existing release version number during the process, this action will replace the existing release with the newly-built one. This can be what you expected... or not. Be cautious.''
A ''Github account'' is mandatory for LuxCore development.


Prerequisite: you must have release a compatible version of External Dependencies (LuxCoreDeps) before building Python Wheels.
Following accounts are also recommended, in order to get in touch with LuxCore community:
* [https://forums.luxcorerender.org/ LuxCoreRender Forums]
* [https://discord.gg/chPGsKV LuxCoreRender Discord channel]


To build a new release of Python Wheels:
== General Developer Workflow ==


{{#mermaid: flowchart TD
To compile and verify LuxCore component modifications in development process, there are 3 stages:
* Local plain build: only run build system (cmake) on developer computer
* Local Continuous Integration build: run full Continuous Integration workflow on developer computer
* Server Continuous Integration build: run full Continuous Integration workflow on server


repo(Go to <a href='https://github.com/LuxCoreRender/LuxCore'><code>LuxCoreRender/LuxCoreDeps</code></a> Github repository)
<i>Local plain build</i> is generally faster than other builds and does not require to commit changes beforehand. Plain build installs a minimal build configuration (dependencies), but heavily relies on computer build environment for the rest (build tools, system libraries etc.). As a counterpart, it builds only for the platform where it is executed and can be significantly altered by local conditions (build tools local versions, dependency local versions etc.), thus not fully guaranteeing that final publish workflow will succeed.
action(Open <a href='https://github.com/LuxCoreRender/LuxCore/actions'><code>Actions</code></a>  menu)
select(Select <a href='https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-releaser.yml'><code>LuxCore Wheels Releaser</code></a> workflow)
runwf(Click on <code>Run workflow</code> drop-down button, in the upper-right corner)
choice(Set branch, release version and other options)
validate(Click <code>Run workflow</code>)
wait(Wait for build to complete)
release(Open the <a href='https://github.com/LuxCoreRender/LuxCore/releases'>LuxCore release folder</a> in Github, </br> find the newly-built release, which is in Draft mode, </br> and edit it)
complete(Add relevant information: change log, comments on version etc.)
dedraft(Unset Draft mode)
update(Click <code>Publish release</code>/<code>Update release</code> to finalize)


repo-->action
<i>LocalCI </i> is generally faster than Server build, but slower than local plain build. CI workflow installs a complete build environment (compilers, sdk, build tools, containers, dependencies etc.) before running build system. It does not require to commit changes beforehand. However, it builds only for the platform where it is executed, thus not fully guaranteeing that final publish workflow will succeed.
action-->select
select-->runwf
runwf-->choice
choice-->validate
validate-->wait
wait:::WAIT-- if success -->release
release-->complete
complete-->dedraft
dedraft-->update


classDef WAIT fill:#ddd;
<I>Server CI</i>, if successful, should guarantee success when publishing, as it occurs in conditions similar to release workflow (all platforms, complete build environment). As a counterpart, it is generally significantly slower than local builds and also implies to commit and push changes beforehand, even if the code is in unstable state.


}}
The recommended workflow thus strives to make the most of all stages:


==== Deploy ====
{{#mermaid:


As far as wheels are concerned, in addition to building them, it is also necessary to upload them to [https://pypi.org/ PyPi] ("deploy" them), so that they are available for [https://pypi.org/project/pip pip] installation.
%%{ init: {
  'theme': 'neutral',
  'flowchart': { 'curve': 'monotoneX' }
  }
}%%


Prerequisite: you must have released a set of Python Wheels beforehand.
flowchart TD;


{{#mermaid: flowchart TD
fork(Fork Source repository in your own Github space)
clone("Clone your fork locally</br><code>git clone...</code>")
devel(Make modifications in your local fork)
localBuild("<b>Run local plain build</b></br><code>cmake</code>, <code>make</code>...")
localCI("<b>Run local CI</b></br><code>act</code>")
git("Commit, Rebase & Push</br><code>git ...</code>")
serverCI(<b>Run server CI</b></br><code>Github workflow</code>)
PR(PR your changes to Source repository)


repo(Go to <a href='https://github.com/LuxCoreRender/LuxCore'><code>LuxCoreRender/LuxCoreDeps</code></a> Github repository)
fork-->clone
action(Open <a href='https://github.com/LuxCoreRender/LuxCore/actions'><code>Actions</code></a> menu)
clone-->devel
select(Select <a href='https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-publisher.yml'><code>LuxCore Python Wheels Publisher</code></a> workflow)
devel-->localBuild
runwf(Click on <code>Run workflow</code> drop-down button, in the upper-right corner)
localBuild-->localCI
choice(Select Branch, set the Tag to publish and uncheck Publish on Pypi Test if needed)
localCI-- "<i>Success</br>(including debug & tests)</i>" -->git
validate(Click on <code>Run workflow</code> button)
git-->serverCI
wait(Wait for deployment to complete)
serverCI-- <i>Success</i> -->PR


localBuild & localCI & serverCI ----> |<i>Failure</i>| devel


 
classDef BUILD fill:#acc,stroke-width:4px;
repo-->action
class localBuild BUILD;
action-->select
class localCI BUILD;
select-->runwf
class serverCI BUILD;
runwf-->choice
choice-->validate
validate-->wait:::WAIT
 
classDef WAIT fill:#ddd;


}}
}}


=== Samples ===
This is the common process between all components. Each step will then be adapted to each component, as explained below.


As we plan to replace Samples by Python tools, there is no Releaser workflow, just a Builder one. Releases have to be created manually, on the basis of Builder's artifacts.
Comments:
* Running local CI relies on [https://github.com/nektos/act <code>nektos/act</code>]
* Rebase must be made against upstream directory (<code>git pull --rebase upstream <branch></code>)before pushing
* Push must be made towards developer own fork. Caveat: do not try to push directly to upstream


{{#mermaid: flowchart TD
== Per-target details ==


repo(Go to <a href='https://github.com/LuxCoreRender/LuxCore'><code>LuxCoreRender/LuxCore</code></a> Github repository)
=== External Dependencies ===
action(Open <a href='https://github.com/LuxCoreRender/LuxCore/actions'><code>Actions</code></a>  menu)
select(Select <a href='https://github.com/LuxCoreRender/LuxCore/actions/workflows/sample-builder.yml'><code>LuxCore Samples Builder</code></a> workflow)
runwf(Click on <code>Run workflow</code> button, in the upper-right corner)
choice(Set branch)
validate(Click <code>Run workflow</code>)
wait(Wait for build to complete)


==== Local Plain Build ====
<i>This feature is not available yet. Meanwhile, you may use Local CI instead.</i>


==== Local Continuous Integration ====


repo:::TARGETSAMPLES-->action
Run <code>act</code> on workflows located in <code>.github/workflows</code> folder, for instance <code>.github/workflows/build.yml</code> workflow.</br>
action:::TARGETSAMPLES-->select
select:::TARGETSAMPLES-->runwf
runwf:::TARGETSAMPLES-->choice
choice:::TARGETSAMPLES-->validate
validate:::TARGETSAMPLES-->wait:::WAIT


classDef WAIT fill:#ddd;
For more convenience, a wrapper script is also provided in <code>LuxCoreDeps</code> repository: <code>utils/debug.sh</code> (Linux only, at the moment).
classDef SAMPLES fill:#ffffdd,font-weight:bold,font-size:1.1rem,stroke:#999999;
classDef TARGETSAMPLES fill:#eeeecc,stroke:#999999;


==== Server Continuous Integration ====


}}
This process relies on the following Github workflow: [https://github.com/LuxCoreRender/LuxCoreDeps/actions/workflows/checker.yml <code>LuxCore Dependency Checker</code>].
</br>Important points:
* Your changes must have been committed and pushed to repository before.
* The Checker workflow is actually also triggered on push event, so you may not have to run it by yourself


== Developer Workflows ==
If build succeeds, you will find the expected outputs in Artifacts section of the action run.


=== LuxCore, Samples & Python Wheels ===


=== Prerequisites ===
==== Local Plain Build ====


You will need various tools installed and referenced in the PATH.
This method internally relies on <code>cmake</code> but, for more convenience, everything has been wrapped into <code>make</code>-like statements.


First, ensure you have a suitable toolchain:
First, we assume you have correctly completed the [[#Prerequisites_2 | Prerequisites]].


{| class="wikitable" style="margin:auto"
Second, we also assume you've already git-cloned LuxCore on your local machine, opened a terminal, changed directory to LuxCore folder and git-checked out the branch that you want to build (+ for Windows: initialized build environment with <code>vcvarsall.bat</code>)
|-
! OS !! Compiler !! Minimal version
|-
| Windows || MSVC || 194x latest version
|-
| Linux|| gcc || 14
|-
| MacOS Intel|| XCode || 15.2
|-
| MacOS Arm || XCode || 15.4
|}


''Toolchain compliance is a very sensitive condition for a successful build. Before filing any claim, please check that your toolchain complies with the above requirements.''
From here on, just enter the magic words:
<pre>
make deps
make
</pre>


Then ensure the following software is also installed and available in your PATH:
This should retrieve and install LuxCore dependencies, configure the project and run the build. That's all!


* [https://git-scm.com/ Git]
* [https://cli.github.com/ Github CLI] (optional but recommended - for dependency signature checking)
* [https://www.python.org/ Python 3] - any [https://devguide.python.org/versions/ currently supported version]
* [https://conan.io Conan] (pip install conan)
* [https://cmake.org/ CMake]
* [https://github.com/nektos/act nektos/act] (optional but recommended - for local build based on Github workflows)


For Windows, ensure the command line is configured for building (vcvarsall.bat).
For subsequent builds, if no modification have been made to dependencies, you'll just have to re-enter: <pre>make</pre>


=== External Dependencies ===


To build external dependencies in development process, there are 2 options:
If you want to target a specific component, you can also use the following verbs:
* Server build: run workflow on server, but without creating any release
* <code>make luxcore</code>
* Local build: run workflow on local computer (user PC)
* <code>make pyluxcore</code>
* <code>make luxcoreui</code>
* <code>make luxcoreconsole</code>


Local build is generally faster than server build and does not require to commit changes beforehand. However, it builds only for the platform where it is executed and can be slightly altered by local conditions (build tools local versions etc.).
Build type can be selected by setting environment variable <code>LUX_BUILD_TYPE</code>. Valid values are 'Release' (default) and 'Debug'.


Recommended workflow is thus:


{{#mermaid:  
You can also build Doxygen documentation with the following statement:
* <code>make doc</code>


%%{ init: {
  'theme': 'neutral',
  'flowchart': { 'curve': 'monotoneX' }
  }
}%%


flowchart TD;
And you can remove build outputs with:
* <code>make clean</code>
* <code>make clear</code>
The latter will remove everything and will require to reinstall dependencies (<code>make deps</code>).


fork(Fork LuxCoreRender/LuxCoreDeps in your own Github repository)
clone(Clone your fork locally)
devel(Make modifications in your local fork)
act("Build locally with <code>act</code></br>(optional but recommended)")
commit(Commit changes)
push(Push to your Github fork)
action(Build on server with <code>Github workflow</code>)
PR(PR your changes to LuxCoreRender LuxCoreDeps)


fork-->clone
Starting from version 2.11, this method can build a "test wheel". This is a simplified wheel, only for current platform and current Python version, for testing purpose. The verb is:
clone-->devel
* <code>make wheel-test</code>
devel-->act
act-- success -->commit
commit-->push
push-->action
action-- success -->PR


act & action ----> |failure| devel
style act stroke-dasharray: 5 5


}}
By default, this method builds a release version, striped from all symbols. If you want to build a debug version, enter <code>export LUX_BUILD_TYPE=Debug</code> before any <code>make ...</code> command.


==== Local Continuous Integration ====


==== Local build ====
This method is complementary to Local plain build, with the following additional capabilities:
This build relies on [https://github.com/nektos/act nektos/act]. Thanks to this very smart tool, it is possible Github Workflows locally. See install instructions on their website.
* test the LuxCore build in an environment close to the release environment (manylinux container, for instance)
* can build wheels
* debug any changes made to the build workflow


After installation, <code>act</code> can be executed directly on command line: call it to run workflows located in <code>.github/workflows</code> folder.</br>
It consists in running Github workflows locally, via <code>act</code>. Github workflows are located in <code>.github/workflows</code> folder.
For more convenience, a wrapper script is also provided in <code>LuxCoreDeps</code> repository: <code>utils/debug.sh</code> (Linux only, at the moment).


<code>act</code> invokation can be made on command line but, for convenience, a wrapper script is provided: <code>build-helpers/debug/debug_wheels.sh</code>.


==== Server build ====
<i>Caveat: this build can produce wheels, but only for the host platform (no cross-compiling).</i>


The process is nearly the same as publisher build, but the release creation steps are skipped. For that purpose, another Github workflow is executed: ''LuxCore Dependency Checker''.</br>
==== Server Continuous Integration ====
Please note that your changes must have been committed to repository beforehand.


{{#mermaid: flowchart TD
The process is nearly the same as publisher build, but the release steps are skipped. For that purpose, another Github workflow has to be executed: [https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-builder.yml <code>LuxCore Python Wheels Builder</code>].
 
repo(Go to <a href='https://github.com/LuxCoreRender/LuxCoreDeps'><code>LuxCoreRender/LuxCoreDeps</code></a> Github repository)
action(Open <a href='https://github.com/LuxCoreRender/LuxCoreDeps/actions'><code>Actions</code></a>  menu)
select(Select <a href='https://github.com/LuxCoreRender/LuxCoreDeps/actions/workflows/check.yml'><code>LuxCore Dependency Checker</code></a> workflow)
runwf(Click on <code>Run workflow</code> button, in the upper-right corner)
wait(Wait for build to complete)


Please note that your changes must have been committed to repository before running the workflow.


If build succeeds, you will find the expected outputs in Artifacts section of the action run.


repo:::DEPS-->action
= Publisher Workflows =
action:::DEPS-->select
select:::DEPS-->runwf
runwf:::DEPS-->wait:::WAIT
 
classDef DEPS fill:#e4d8ff,stroke:#9488af;
classDef WAIT fill:#ddd;
 
}}
 
If build succeeds, you will find the expected outputs in Artifacts section of the action run.


=== LuxCore, Samples & Python Wheels ===
== Prerequisites ==


To build LuxCore, Samples & Python in a development process, there are 3 methods:
For these workflows, a Github account is required, with extended rights on the source repositories.
* Local build - make wrapper
* Local build - Github workflows
* Server build: run workflow on server, but without creating any release
</br>
Pros and cons of each method are summarized below:


{| class="wikitable" style="margin:auto"
== General Workflow ==
|-
! Method !! Description !! Pros !! Cons
|-
| '''Local build - make wrapper''' || Execute cmake statements via make wrapper || Very fast (incremental). Do not require to commit changes beforehand. Can build Samples as well || Affected by local conditions (tool versions, Python version, libc version, system deps...). Build only for local platform. Cannot build Python wheels (just LuxCore and Samples)
|-
| '''Local build - act''' || Execute Github workflow locally with nektos/act || Fast. More robust to local conditions (containerized). Do not require to commit changes beforehand. Can build Python Wheel || Slower than make wrapper (rebuild all every time). Build only for local platform. Can be slightly affected by local conditions
|-
| '''Server build''' || Execute Github workfow on server, without issuing a release || Build in Release environment, for all platforms and all Python versions || Slow. Requires to commit changes beforehand.
|}


Therefore the recommended workflow is:
The general workflow for publishing is in 3 parts:


{{#mermaid:  
{{#mermaid:  
Line 467: Line 406:
   }
   }
}%%
}%%
flowchart TD;
flowchart TD;
subgraph Build["<center><b>BUILD & RELEASE DRAFT </br>(Run Releaser workflow)</b></center>"]
padBuild(" "):::INVISIBLE
repo("Go to <code>[Source repository]</code> on Github and open <code>Actions menu</code>")
select("Select <code>[Releaser workflow]</code>")
runwf(Click on <code>Run workflow</code> button, in the upper-right corner)
branch(Select <code>Branch</code> in drop-down menu)
options(Set workflow options)
validate(Click <code>Run workflow</code>)
wait(Wait for build to complete)
end


fork(Fork LuxCoreRender/LuxCore in your own Github repository)
subgraph Release["<center><b>MAKE FINAL RELEASE</br>(Switch release to final status)</b></center>"]
clone(Clone your fork locally)
padRelease(" "):::INVISIBLE
devel(Develop in your local fork)
release("Open <code>[Release folder]</code>,</br> find the newly-built release (in Draft mode) </br> and edit it")
make(Build locally with <code>make-wrapper</code>)
complete(Add relevant information: change log, comments on version etc.)
act("Build locally with <code>act</code></br>(optional but recommended)")
dedraft(Unset <code>Draft</code> checkbox)
commit(Commit changes)
update(Click <code>Publish release</code>/<code>Update release</code> to finalize)
push(Push to your Github fork)
end
action(Build on server with <code>Github workflow</code>)
PR(PR your changes to LuxCoreRender LuxCore)


fork-->clone
subgraph Deploy["<center><b>PUBLISH</br>(Run Publisher workflow)</b></center>"]
clone-->devel
padDeploy(" ")
devel-->make
repoDeploy("Go to <code>[Source repository]</code> on Github and open <code>[Actions menu]</code>")
make-- success -->act
selectDeploy("Select <code>[Publisher workflow]</code>")
act-- success -->commit
runwfDeploy("Click on <code>Run workflow</code> drop-down button, in the upper-right corner")
commit-->push
choiceDeploy("Select <code>Branch</code>, set the <code>Tag to publish</code> and uncheck <code>Publish on Pypi Test</code> if needed")
push-->action
validateDeploy("Click on <code>Run workflow</code> button")
action-- success -->PR
waitDeploy("Wait for deployment to complete")
end


make & act & action ----> |failure| devel
padBuild---repo
style act stroke-dasharray: 5 5
repo-->select
select-->runwf
runwf-->branch
branch-->options
options-->validate
validate-->wait:::WAIT
linkStyle 0 stroke:#fff,stroke-width:0px,color:blue;


}}
padRelease---release
release-->complete
complete-->dedraft
dedraft-->update
linkStyle 7 stroke:#fff,stroke-width:0px,color:blue;


padDeploy:::INVISIBLE---repoDeploy
repoDeploy-->selectDeploy
selectDeploy-->runwfDeploy
runwfDeploy-->choiceDeploy
choiceDeploy-->validateDeploy
validateDeploy-->waitDeploy:::WAIT
linkStyle 11 stroke:#fff,color:blue,stroke-width:0px;


==== Local build - make wrapper ====


This is the main method for development/debug cycles. After having cloned LuxCoreRender/LuxCore, open a command line, cd to LuxCore folder, check out the branch that you want to build and enter this magic statement:
Build-- <i>if success</i> -->Release
Release-- <i>if success & if needed</i> -->Deploy


<code>
classDef WAIT fill:#ddd;
'''make deps && make'''
classDef DEPS fill:#e4d8ff,stroke:#9488af;
</code>
classDef INVISIBLE fill:#ffffffff,stroke:#ffffffff,stroke-width:0px,font-color:#ffffffff;
</br>
classDef OPTIONAL stroke-dasharray: 5 5;


(Caveat: on Windows, build environment must have been initialized.)
class Deploy OPTIONAL;


This should retrieve and install LuxCore dependencies, configure the project and trigger the build. Note that, behind the scene, most of work is done by cmake and Python.
%% Note: padXXX is a poor hack to create a top-margin in subgraph...
In subsequent calls, if no modification have been made to dependencies, you'll just have to enter <code>make</code>
}}


If you want to target a specific component, you can also use the following verbs:
A few comments:
* <code>make luxcore</code>
* <code>[bracketed tokens]</code> refer to data specific to each component, which will be detailed below.
* <code>make pyluxcore</code>
* Publish part is optional, it will depends on the target
* <code>make luxcoreui</code>
* ''Caveat: if you provide an already-existing release version number during the process, the workflow will replace the existing release with the newly-built one. This can be what you expected... or not. Be cautious.''
* <code>make luxcoreconsole</code>


Build type can be selected by setting environment variable <code>LUX_BUILD_TYPE</code>. Valid values are 'Release' (default) and 'Debug'.
== Per-target details ==


You can also build Doxygen documentation with the following verb:
=== External Dependencies ===
* <code>make doc</code>


And you can remove build outputs with the following verbs:
* Source repository: [https://github.com/LuxCoreRender/LuxCoreDeps <code>LuxCoreRender/LuxCoreDeps</code>]
* <code>make clean</code>
* Releaser workflow: [https://github.com/LuxCoreRender/LuxCoreDeps/actions/workflows/release.yml <code>LuxCore Dependency Releaser</code>]
* <code>make clear</code>
* Release folder: [https://github.com/LuxCoreRender/LuxCoreDeps/releases <code>Releases</code>]
The latter will remove everything and will require to reinstall dependencies (<code>make deps</code>).
* Publisher workflow: ∅


==== Local build - act ====
As a byproduct, External Dependencies are not intended to be deployed, so no Publisher workflow is provided.


This method is complementary to Local build - make wrapper.
=== Python Wheels ===


It allows:
* to test the LuxCore build in an environment close to the release environment (manylinux container)
* to build wheels locally
* to debug any changes made to the build workflow


It consists in running the Github workflow locally, via <code>act</code>. <code>act</code> invokation can be made on command line but, for convenience, a wrapper script is provided: <code>build-helpers/debug/debug_wheels.sh</code>.


==== Server build - Github workflows ====
As far as wheels are concerned, in addition to building them, it is also necessary to upload them to [https://pypi.org/ PyPi] ("publish" them), so that they are available for [https://pypi.org/project/pip pip] installation.


The process is nearly the same as release build, but the release creation steps are skipped. For that purpose, another Github workflow is executed: ''LuxCore Wheels Builder''.</br>
* Source repository: [https://github.com/LuxCoreRender/LuxCore <code>LuxCoreRender/LuxCore</code>]
Please note that your changes must have been committed to repository before running the workflow.
* Releaser workflow: [https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-releaser.yml <code>LuxCore Wheels Releaser</code>]
 
* Release folder: [https://github.com/LuxCoreRender/LuxCore/releases <code>Releases</code>]
<!-- Debug -->
* Publisher workflow: [https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-publisher.yml <code>LuxCore Wheels Publisher</code>]
{{#mermaid: flowchart TD
 
repo(Go to <a href='https://github.com/LuxCoreRender/LuxCore'><code>LuxCoreRender/LuxCoreDeps</code></a> Github repository)
action(Open <a href='https://github.com/LuxCoreRender/LuxCore/actions'><code>Actions</code></a>  menu)
select(Select <a href='https://github.com/LuxCoreRender/LuxCore/actions/workflows/wheel-builder.yml'><code>LuxCore Python Wheels Builder</code></a> workflow)
runwf(Click on <code>Run workflow</code> drop-down button, in the upper-right corner)
choice(Select Branch and Build type)
validate(Click on <code>Run workflow</code> button)
wait(Wait for build to complete)


''Prerequisite: you must have released a compatible version of External Dependencies (LuxCoreDeps) before building Python Wheels.''


=== Samples ===


repo-->action
As we plan to replace Samples by equivalent Python tools and thus deprecate them, the workflow has been limited to the Release part (no publish).
action-->select
select-->runwf
runwf-->choice
choice-->validate
validate-->wait:::WAIT


classDef WAIT fill:#ddd;
* Source repository: [https://github.com/LuxCoreRender/LuxCore <code>LuxCoreRender/LuxCore</code>]
* Releaser workflow: [https://github.com/LuxCoreRender/LuxCore/actions/workflows/sample-releaser.yml <code>LuxCore Samples Releaser</code>]
* Release folder: [https://github.com/LuxCoreRender/LuxCore/releases <code>Releases</code>]
* Publisher workflow: ∅


}}
=== BlendLuxCore ===


If build succeeds, you will find the expected outputs in Artifacts section of the action run.
<i>TODO</i>


== Package Maintainers ==
=== PyLuxCoreTools ===


First of all: if you plan to package LuxCore for your distribution, thank you very much.
<i>TODO</i>
 
Most likely, your interest should be in Samples, as LuxCore's Dependencies are of no use to anyone but us (and are not something we'd offer packaging support on anyway) and the Wheels are already packaged and deployed on PyPi. Thus we shall focus on luxcoreui and luxcoreconsole.
 
To build Samples, we would recommend to use <code>make deps && make luxcoreui && make luxcoreconsole</code> invokation as described in [[#Developper_Workflows]].
You will find binaries in out/install/Release.
As an alternative, you can also run nektos/act on .github/workflows/sample-builder.yml workflow. The benefit of this option is that build occurs in manylinux_2_28 container, thus enlarging compatibility (glibc 2.28+).
All LuxCore dependencies are provided by LuxCoreDeps, except system ones (glibc, opengl, Xorg...). Except Nvidia nvrtc, provided dependencies are all built from sources by LuxCoreDeps, in the same build environment as the Wheels (including manylinux_2_28 container), and their build options are carefully selected.
 
Most of dependencies are linked statically to LuxCore. Unless your distribution policy prevents it, we would recommend not to substitute those static deps with the ones provided by your distro.
Apart from system dependencies, the only shared libraries are:
- OneAPI TBB
- OneAPI embree3
- Nvidia nvrtc
- OpenImageDenoise
All those libraries are provided along with LuxCore. RPATH are set in LuxCore to search for the provided versions first, at runtime. Thus we would also recommend to keep those versions, instead of your distro ones.
 
At any rate, if you should encounter any difficulty, please contact us.


= Version Management =
= Version Management =


=== Semantic Versioning ===
== Semantic Versioning ==


Starting for LuxCore version 2.10.0, we require that LuxCoreRender applies [https://semver.org/ Semantic Versioning]. Please strictly enforce this rule when incrementing version numbers.
Starting for LuxCore version 2.10.0, we require that LuxCoreRender applies [https://semver.org/ Semantic Versioning]. Please strictly enforce this rule when incrementing version numbers.
Line 596: Line 527:
Remark: component versions does not have to be synced. For instance, in strict application of the standard, the newly created LuxCoreDeps will start at 1.0.0, while at the same time LuxCore is already at 2.10.0
Remark: component versions does not have to be synced. For instance, in strict application of the standard, the newly created LuxCoreDeps will start at 1.0.0, while at the same time LuxCore is already at 2.10.0


=== Dependency Version pointer ===
== Dependency Version pointer ==


To make LuxCore know which version of dependencies it must be built against, we use a pointer.
To make LuxCore know which version of dependencies it must be built against, we use a pointer.


This pointer is stored in the file <code>build-helpers/build-settings.json</code>, under the path <code>Dependencies/release</code>.
This pointer is stored in the file <code>build-helpers/build-settings.json</code>, under the path <code>Dependencies/release</code>.
Make sure you keep it up-to-date when you upgrade LuxCore components.

Latest revision as of 18:13, 10 August 2025

This document describes the various processes involved in building LuxCoreRender, starting with version 2.10 “Back-on-track”.

LuxCoreRender's build system has been significantly modified for version 2.10. This document therefore renders obsolete all previous documents related to the compilation of older versions (<=2.9).


TL;DR If you are just interested in a quick, local (but platform limited) compilation of LuxCore C++ samples or Python wheels, you may want to jump directly to LuxCore, Samples & Python Wheels.


Audience

This document is intended for:

  • LuxCoreRender administrators in charge of releasing the various LuxCoreRender end-products;
  • Developers wishing to contribute to the project;
  • External package maintainers wishing to integrate all or part of LuxCoreRender into a distribution;

This document assumes that the reader is skilled in the following areas:

Familiarity with Conan dependency manager may also help.


This document is not intended for end-users without any knowledge about application building. Such users are invited to look for already compiled binaries on the official Download page.

Build Targets

Synoptic

LuxCoreRender contains several build targets, below represented with dependency links:

Target Source Repository Content
External Dependencies LuxCoreDeps A bundled Conan cache populated with LuxCoreRender external dependency binaries, built from sources. Please refer to LuxCoreDeps README for more information.
LuxCore LuxCore LuxCore core binaries, in the form of static and shared libraries: luxcore.so, luxcore_static.lib, luxcore.dll etc.
Samples LuxCore Sample C++ programs, illustrating luxcore use, namely luxcoreconsole and luxcoreui
Python Wheels LuxCore Python bindings of core binaries, in the form of Python wheels (1 per pair Platform/Python version).
As a byproduct, a Pythonized version of LuxCore shared library is also built (pyluxcore.so).
Python Plugins BlendLuxCore etc. Plugins to expose LuxCore in various external applications, notably Blender.
Written in Python and relying on Python Wheels as runtime dependency.
PyLuxCoreTools LuxCore Python LuxCore Tools: set of command line tools based on pyluxcore, written in Python.

Targeted platforms

LuxCoreRender aims at being available on the following 4 platforms:

  • Linux (glibc 2.28+)
  • Windows
  • MacOS Intel (>=10.15)
  • MacOS Arm (>=12.0)

For Python-related targets, LuxCoreRender aims at being declined for all Python versions supported at a given time.

Workflow Types

We distinguish between two types of build workflows:

  • Developer workflows
  • Publisher workflows


Developer workflows are designed for development, debugging and test steps.

Their purpose is to allow development and verify the code is ready for publication.


Publisher workflows are designed to be used by LuxCoreRender administrators to publish a new release of one or more LuxCoreRender components.

They exclusively take place in a CI/CD Github pipeline.

These workflows are not designed for debugging or testing the underlying code. Execution of these workflows assumes that the development phase has been correctly completed and the underlying code is ready for final build and release. See Developer Workflows to learn how to get code ready for final build.

Running these workflows require users to be granted of extended rights on LuxCoreRender repositories.

Developer Workflows

Prerequisites

Tools

You will need several build tools installed and referenced in the PATH.

First, ensure you have a suitable toolchain:

OS Compiler Minimal version
Windows MSVC 194x latest version
Linux gcc 14
MacOS Intel XCode + llvm + libomp llvm >= 20
(brew install llvm && brew install libomp)
MacOS Arm XCode + llvm + libomp llvm >= 20
(brew install llvm && brew install libomp)

Toolchain compliance is a very sensitive condition for a successful build. Before filing any claim, please check that your toolchain complies with the above requirements.


Then ensure the following software is also installed and available in your PATH:


Platform specifics:

  • For Windows, ensure the command line is configured for building (vcvarsall.bat).
  • For Linux, you'll need pkgconfig in addition.
  • For MacOS, you'll need to ensure CC and CXX point to LLVM clang/clang++

Accounts

A Github account is mandatory for LuxCore development.

Following accounts are also recommended, in order to get in touch with LuxCore community:

General Developer Workflow

To compile and verify LuxCore component modifications in development process, there are 3 stages:

  • Local plain build: only run build system (cmake) on developer computer
  • Local Continuous Integration build: run full Continuous Integration workflow on developer computer
  • Server Continuous Integration build: run full Continuous Integration workflow on server

Local plain build is generally faster than other builds and does not require to commit changes beforehand. Plain build installs a minimal build configuration (dependencies), but heavily relies on computer build environment for the rest (build tools, system libraries etc.). As a counterpart, it builds only for the platform where it is executed and can be significantly altered by local conditions (build tools local versions, dependency local versions etc.), thus not fully guaranteeing that final publish workflow will succeed.

LocalCI is generally faster than Server build, but slower than local plain build. CI workflow installs a complete build environment (compilers, sdk, build tools, containers, dependencies etc.) before running build system. It does not require to commit changes beforehand. However, it builds only for the platform where it is executed, thus not fully guaranteeing that final publish workflow will succeed.

Server CI, if successful, should guarantee success when publishing, as it occurs in conditions similar to release workflow (all platforms, complete build environment). As a counterpart, it is generally significantly slower than local builds and also implies to commit and push changes beforehand, even if the code is in unstable state.

The recommended workflow thus strives to make the most of all stages:

This is the common process between all components. Each step will then be adapted to each component, as explained below.

Comments:

  • Running local CI relies on nektos/act
  • Rebase must be made against upstream directory (git pull --rebase upstream <branch>)before pushing
  • Push must be made towards developer own fork. Caveat: do not try to push directly to upstream

Per-target details

External Dependencies

Local Plain Build

This feature is not available yet. Meanwhile, you may use Local CI instead.

Local Continuous Integration

Run act on workflows located in .github/workflows folder, for instance .github/workflows/build.yml workflow.

For more convenience, a wrapper script is also provided in LuxCoreDeps repository: utils/debug.sh (Linux only, at the moment).

Server Continuous Integration

This process relies on the following Github workflow: LuxCore Dependency Checker.
Important points:

  • Your changes must have been committed and pushed to repository before.
  • The Checker workflow is actually also triggered on push event, so you may not have to run it by yourself

If build succeeds, you will find the expected outputs in Artifacts section of the action run.

LuxCore, Samples & Python Wheels

Local Plain Build

This method internally relies on cmake but, for more convenience, everything has been wrapped into make-like statements.

First, we assume you have correctly completed the Prerequisites.

Second, we also assume you've already git-cloned LuxCore on your local machine, opened a terminal, changed directory to LuxCore folder and git-checked out the branch that you want to build (+ for Windows: initialized build environment with vcvarsall.bat)

From here on, just enter the magic words:

make deps
make

This should retrieve and install LuxCore dependencies, configure the project and run the build. That's all!


For subsequent builds, if no modification have been made to dependencies, you'll just have to re-enter:

make


If you want to target a specific component, you can also use the following verbs:

  • make luxcore
  • make pyluxcore
  • make luxcoreui
  • make luxcoreconsole

Build type can be selected by setting environment variable LUX_BUILD_TYPE. Valid values are 'Release' (default) and 'Debug'.


You can also build Doxygen documentation with the following statement:

  • make doc


And you can remove build outputs with:

  • make clean
  • make clear

The latter will remove everything and will require to reinstall dependencies (make deps).


Starting from version 2.11, this method can build a "test wheel". This is a simplified wheel, only for current platform and current Python version, for testing purpose. The verb is:

  • make wheel-test


By default, this method builds a release version, striped from all symbols. If you want to build a debug version, enter export LUX_BUILD_TYPE=Debug before any make ... command.

Local Continuous Integration

This method is complementary to Local plain build, with the following additional capabilities:

  • test the LuxCore build in an environment close to the release environment (manylinux container, for instance)
  • can build wheels
  • debug any changes made to the build workflow

It consists in running Github workflows locally, via act. Github workflows are located in .github/workflows folder.

act invokation can be made on command line but, for convenience, a wrapper script is provided: build-helpers/debug/debug_wheels.sh.

Caveat: this build can produce wheels, but only for the host platform (no cross-compiling).

Server Continuous Integration

The process is nearly the same as publisher build, but the release steps are skipped. For that purpose, another Github workflow has to be executed: LuxCore Python Wheels Builder.

Please note that your changes must have been committed to repository before running the workflow.

If build succeeds, you will find the expected outputs in Artifacts section of the action run.

Publisher Workflows

Prerequisites

For these workflows, a Github account is required, with extended rights on the source repositories.

General Workflow

The general workflow for publishing is in 3 parts:

A few comments:

  • [bracketed tokens] refer to data specific to each component, which will be detailed below.
  • Publish part is optional, it will depends on the target
  • Caveat: if you provide an already-existing release version number during the process, the workflow will replace the existing release with the newly-built one. This can be what you expected... or not. Be cautious.

Per-target details

External Dependencies

As a byproduct, External Dependencies are not intended to be deployed, so no Publisher workflow is provided.

Python Wheels

As far as wheels are concerned, in addition to building them, it is also necessary to upload them to PyPi ("publish" them), so that they are available for pip installation.

Prerequisite: you must have released a compatible version of External Dependencies (LuxCoreDeps) before building Python Wheels.

Samples

As we plan to replace Samples by equivalent Python tools and thus deprecate them, the workflow has been limited to the Release part (no publish).

BlendLuxCore

TODO

PyLuxCoreTools

TODO

Version Management

Semantic Versioning

Starting for LuxCore version 2.10.0, we require that LuxCoreRender applies Semantic Versioning. Please strictly enforce this rule when incrementing version numbers.

Remark: component versions does not have to be synced. For instance, in strict application of the standard, the newly created LuxCoreDeps will start at 1.0.0, while at the same time LuxCore is already at 2.10.0

Dependency Version pointer

To make LuxCore know which version of dependencies it must be built against, we use a pointer.

This pointer is stored in the file build-helpers/build-settings.json, under the path Dependencies/release.

Make sure you keep it up-to-date when you upgrade LuxCore components.