Automated Render Benchmarking (Windows Only)

Blender Courses Talk
, ,
#1

Tweaking values and waiting for results repeatedly is pretty boring task. That’s why I was looking for some automated method for measuring render times with different settings.

There is an add-on in Blender called Render: Auto Tile Size, which is supposed to “Estimate and set the tile size that will render the fastest”. Unfortunately this add-on is not working on my system.

I did not manage to find another solution online, so I decided to make my own and this is the result:

benchmark-output
benchmark-output.png377×587 5.47 KB

How it works

Since Blender can render scenes without graphical interface via command line, I created a windows batch file that starts rendering multiple times with different parameters and measures times between start and finish. The batch file creates a temporary file with a python script that changes the tile sizes before each test, then executes blender.exe passing this python script as a parameter.

How to use it

  1. Create new text file in the same directory as your .blend file
  2. Copy and paste the code from bottom of this post to the new text file
  3. On beginning of the code, there are 4 configuration lines.

SET blender_bin="C:/Program Files/Blender Foundation/Blender/blender.exe"
First line points to blender executable file. If you have Blender installed in different directory, change it in the code.

SET "blend_file=BMW27.blend"
Second line points to .blend file that will be rendered. If you want to benchmark with different file as the BMW27.blend from assets to section 5, change it here.

SET "list_CPU=4x4 8x8 16x16 32x32 64x64"
SET "list_GPU=32x32 64x64 128x128 256x256 124x135 248x270 495x540"
Third and forth lines are list of tile sizes that will be tested for CPU and GPU respectively. If you want to add or remove some tests, do it here. If you only want to test CPU or only GPU, keep one of the lists empty (eg. SET "list_GPU="). Keep the formatting - X and Y tile sizes must be separated by letter x and combinations must be separated by space. Also keep the quotes intact.

  1. Save the text file and close it. Name the file as you want and change the extension from .txt to .bat. If you can’t change the extension, you might want to change your Windows settings to show file extensions.
  2. You should now have a file called for example run_benchmark.bat in a directory with your .blend file. Now just run the file and the testing should begin.

How it behaves

  • The command window shows progress before and after each rendering. It does not output anything while rendering.
  • A new text file is created in the same directory in each pass. Name format of this files is DEVICE_XxYSIZE.txt (eg. GPU_128x128.txt). These files contains command line output from Blender, and can be used to further analyze rendering process and/or for troubleshooting.
  • File tmp.py is generated before each pass. It contains a python script that tells Blender what tile size will be used for rendering. This file has no usage for user.
  • File 0001.png is generated in each pass by Blender. This is the image result of rendering. It is overridden in each pass.
  • File output.txt contains the result of benchmarking (the same as is shown in the command line window). It is created for future reference and to keep the information when command line window is accidentally closed.

At the end of benchmark, the directory which originally contained only the .blend file and this .bat file, should now look like this:

benchmark-generated-files
benchmark-generated-files.png549×633 42.3 KB

You can delete any of the files if you don’t need them.

Known Bugs

  • Time calculation WILL BE incorrect when rendering at the end of the day and beginning of another. For example when rendering current pass starts at 23:58:00 and ends at 00:02:00, the result will not be 4 minutes, but a large negative number. In such case, the generated text files with blender output can be examined to find out correct rendering time.
  • I did not put any effort to make it foolproof, so any invalid data will have unexpected results. For example incorrectly formatted list of tile sizes, or invalid path to Blender installation directory.

You can modify and redistribute the code to your heart's content. Please let me know if you find it useful or run into a problem.

THE CODE:

@ECHO OFF

::CONFIG
SET blender_bin="C:/Program Files/Blender Foundation/Blender/blender.exe"
SET "blend_file=BMW27.blend"
SET "list_CPU=4x4 8x8 16x16 32x32 64x64"
SET "list_GPU=32x32 64x64 128x128 256x256 124x135 248x270 495x540"

::SCRIPT START TIME
SET "script_startTime=%time%"

ECHO [ Starting Benchmark ]
ECHO [ Start Time: %script_startTime% ]
ECHO.
ECHO [ Starting Benchmark ] > output.txt
ECHO [ Start Time: %script_startTime% ] >> output.txt
ECHO. >> output.txt

::BENCH CPU
FOR %%a IN (%list_CPU%) DO (
  ECHO Testing CPU %%a...
  ECHO Testing CPU %%a... >> output.txt
  CALL :bench CPU %%a
)

::BENCH GPU
FOR %%a IN (%list_GPU%) DO (
  ECHO Testing GPU %%a...
  ECHO Testing GPU %%a... >> output.txt
  CALL :bench GPU %%a
)

::GO TO END
GOTO :end

:bench
CALL :preparePythonScriptFile %~1 %~2
SET "startTime=%time%"
%blender_bin% --background %blend_file% --python tmp.py --render-frame 1 --render-format PNG --engine CYCLES > %~1_%~2.txt
SET "stopTime=%time%"
CALL :timeDiff diff startTime stopTime
ECHO ...done in %diff% seconds.
ECHO.
ECHO ...done in %diff% seconds. >> output.txt
ECHO. >> output.txt
GOTO :eof

:preparePythonScriptFile
FOR /f "tokens=1* delims=x" %%a in ("%~2") do (
  SET "tile_x=%%a"
  SET "tile_y=%%b"
)
ECHO import bpy > tmp.py
ECHO for scene in bpy.data.scenes: >> tmp.py
ECHO   scene.cycles.device = '%~1' >> tmp.py
ECHO   scene.render.tile_x = %tile_x% >> tmp.py
ECHO   scene.render.tile_y = %tile_y% >> tmp.py
GOTO :eof

:timeDiff
SETLOCAL
CALL :timeToS time1 "%~2"
CALL :timeToS time2 "%~3"
SET /a diff=(time2-time1)
(
  ENDLOCAL
  SET "%~1=%diff%"
  GOTO :eof
)

:timeToS
::### WARNING, enclose the time in " ", because it can contain comma seperators
SETLOCAL EnableDelayedExpansion
FOR /F "tokens=1,2,3,4 delims=:,.^ " %%a IN ("!%~2!") DO (
  SET /a "s=(((30%%a%%100)*60+7%%b)*60+3%%c-42300)"
)
(
  ENDLOCAL
  SET %~1=%s%
  GOTO :eof
)

:end
SET "script_stopTime=%time%"
CALL :timeDiff diff script_startTime script_stopTime
ECHO     [ ALL DONE ]
ECHO     [ End Time: %script_stopTime% ]
ECHO     [ Total Time: %diff% seconds ]
ECHO. 
ECHO     [ ALL DONE ] >> output.txt
ECHO     [ End Time: %script_stopTime% ] >> output.txt
ECHO     [ Total Time: %diff% seconds ] >> output.txt
ECHO. >> output.txt
PAUSE
Rendered Performance Chart - Core i5 / GTX 460

Privacy & Terms