loki.batch.tests.test_scheduler

Specialised test that exercises the bulk-processing capabilities and source-injection mechanism provided by the loki.scheduler and loki.task sub-modules.

Test directory structure

  • projA: - include

    • another_l1.intfb.h

    • another_l2.intfb.h

    • source - another_l1 - another_l2

    • module - header_mod

      • header_type

      • driverA_mod

      • kernelA_mod

      • compute_l1_mod

      • compute_l2_mod

      • driverB_mod

      • kernelB_mod

      • driverC_mod

      • kernelC_mod

      • driverD_mod

      • kernelD_mod

  • projB: - external

    • ext_driver_mod

    • module - ext_kernel_mod

  • projC: - util

    • proj_c_util_mod * routine_one * routine_two

Functions

fixture_config()

Default configuration dict with basic options.

fixture_driverA_dependencies()

fixture_driverB_dependencies()

fixture_frontend()

Frontend to use.

fixture_here()

fixture_loki_69_dir(testdir)

Fixture to write test file for LOKI-69 test.

fixture_proj_typebound_dependencies()

fixture_testdir(here)

test_pipeline_config_compose(config)

Test the correct instantiation of a custom Pipeline object from config.

test_scheduler_cmake_planner(tmp_path, ...)

Test the plan generation feature over a call hierarchy spanning two distinctive projects.

test_scheduler_config_match_item_keys(...)

test_scheduler_cycle(tmp_path, config, ...)

test_scheduler_definitions(testdir, config, ...)

Create a simple task graph and inject type info via definitions.

test_scheduler_dependencies_ignore(tmp_path, ...)

Test multi-lib transformation by applying the DependencyTransformation over two distinct projects with two distinct invocations.

test_scheduler_depths(testdir, config, ...)

test_scheduler_disable_wildcard(testdir, ...)

test_scheduler_empty_config(testdir, ...)

Test that instantiating the Scheduler without config works (albeit it's not very useful) This fixes #373

test_scheduler_enrichment(testdir, config, ...)

test_scheduler_filter_items_file_graph(...)

Ensure that the items list given to a transformation in a file graph traversal is filtered to include only used items

test_scheduler_frontend_args(tmp_path, ...)

Test overwriting frontend options via Scheduler config

test_scheduler_frontend_overwrite(tmp_path, ...)

Test the use of a different frontend via Scheduler config

test_scheduler_graph_blocked(tmp_path, ...)

Create a simple task graph with a single branch blocked:

test_scheduler_graph_config_file(tmp_path, ...)

Create a sub-graph from a branches using a config file:

test_scheduler_graph_multiple_combined(...)

Create a single task graph spanning two projects

test_scheduler_graph_multiple_separate(...)

Tests combining two scheduler graphs, where that an individual sub-branch is pruned in the driver schedule, while IPA meta-info is still injected to create a seamless jump between two distinct schedules for projA and projB

test_scheduler_graph_multiple_separate_enrich_fail(...)

Tests that explicit enrichment in "strict" mode will fail because it can't find ext_driver

test_scheduler_graph_partial(tmp_path, ...)

Create a sub-graph from a select set of branches in single project:

test_scheduler_graph_simple(tmp_path, ...)

Create a simple task graph from a single sub-project:

test_scheduler_ignore_external_item(...)

test_scheduler_indirect_import(frontend, ...)

test_scheduler_interface_dependencies(...)

Ensure that interfaces are treated as intermediate nodes and incur dependencies on the actual procedures

test_scheduler_interface_inline_call(...)

Test that inline function calls declared via an explicit interface are added as dependencies.

test_scheduler_item_dependencies(testdir, ...)

Make sure children are correct and unique for items

test_scheduler_item_successors(testdir, ...)

Test that scheduler.item_successors always returns the original item.

test_scheduler_loki_69(loki_69_dir, tmp_path)

Test compliance of scheduler with edge cases reported in LOKI-69.

test_scheduler_member_routines(tmp_path, ...)

Make sure that transformation processing works also for contained member routines

test_scheduler_missing_files(testdir, ...)

Ensure that strict=True triggers failure if source paths are missing and that strict=False goes through gracefully.

test_scheduler_module_dependencies_unqualified(...)

Ensure dependency chasing is done correctly for unqualified module imports.

test_scheduler_module_dependency(testdir, ...)

Ensure dependency chasing is done correctly, even with surboutines that do not match module names.

test_scheduler_nested_type_enrichment(...)

Make sure that enrichment works correctly for nested types across multiple source files

test_scheduler_pipeline_simple(testdir, ...)

Test processing a Pipeline over a simple call-tree.

test_scheduler_process(testdir, config, ...)

Create a simple task graph from a single sub-project and apply a simple transformation to it.

test_scheduler_process_filter(testdir, ...)

Applies simple kernels over complex callgraphs to check that we only apply to the entities requested and only once!

test_scheduler_scopes(tmp_path, testdir, ...)

Test discovery with import renames and duplicate names in separate scopes

test_scheduler_successors(tmp_path, config, ...)

test_scheduler_traversal_order(tmp_path, ...)

Test correct traversal order for scheduler processing

test_scheduler_typebound(tmp_path, testdir, ...)

Test correct dependency chasing for typebound procedure calls.

test_scheduler_typebound_ignore(tmp_path, ...)

Test correct dependency chasing for typebound procedure calls with ignore working for typebound procedures correctly.

test_scheduler_typebound_inline_call(...)

test_scheduler_unqualified_imports(config)

Test that only qualified imports are added as children.

test_transformation_config(config)

Test the correct instantiation of Transformation objecst from config

test_transformation_config_external_with_dimension(...)

Test instantiation of Transformation from config with Dimension argument.

Classes

VisGraphWrapper(path)

Testing utility to parse the generated callgraph visualisation.

fixture_here()
fixture_testdir(here)
fixture_config()

Default configuration dict with basic options.

fixture_frontend()

Frontend to use.

Not parametrizing the tests as the scheduler functionality should be independent from the specific frontend used. Cannot use OMNI for this as not all tests have dependencies fully resolved.

fixture_driverA_dependencies()
fixture_driverB_dependencies()
fixture_proj_typebound_dependencies()
class VisGraphWrapper(path)

Bases: object

Testing utility to parse the generated callgraph visualisation.

property nodes
property edges
test_scheduler_enrichment(testdir, config, frontend, tmp_path)
test_scheduler_empty_config(testdir, frontend, tmp_path)

Test that instantiating the Scheduler without config works (albeit it’s not very useful) This fixes #373

test_scheduler_graph_simple(tmp_path, testdir, config, frontend, driverA_dependencies, with_file_graph, with_legend, seed)

Create a simple task graph from a single sub-project:

projA: driverA -> kernelA -> compute_l1 -> compute_l2

–> another_l1 -> another_l2
test_scheduler_graph_partial(tmp_path, testdir, config, frontend, seed)

Create a sub-graph from a select set of branches in single project:

projA: compute_l1 -> compute_l2

another_l1 -> another_l2

test_scheduler_graph_config_file(tmp_path, testdir, frontend)

Create a sub-graph from a branches using a config file:

projA: compute_l1 -> compute_l2

another_l1 -> another_l2

test_scheduler_graph_blocked(tmp_path, testdir, config, frontend, seed)

Create a simple task graph with a single branch blocked:

projA: driverA -> kernelA -> compute_l1 -> compute_l2

X –> <blocked>

test_scheduler_definitions(testdir, config, frontend, seed, tmp_path)

Create a simple task graph and inject type info via definitions.

projA: driverA -> kernelA -> compute_l1 -> compute_l2

<header_type>
–> another_l1 -> another_l2
test_scheduler_process(testdir, config, frontend, seed, tmp_path)

Create a simple task graph from a single sub-project and apply a simple transformation to it.

projA: driverA -> kernelA -> compute_l1 -> compute_l2
<driver> <kernel>

–> another_l1 -> another_l2 <driver> <kernel>
test_scheduler_process_filter(testdir, config, frontend, seed, tmp_path)

Applies simple kernels over complex callgraphs to check that we only apply to the entities requested and only once!

projA: driverE_single -> kernelE -> compute_l1 -> compute_l2

–> ghost_busters
test_scheduler_graph_multiple_combined(tmp_path, testdir, config, driverB_dependencies, frontend)

Create a single task graph spanning two projects

projA: driverB -> kernelB -> compute_l1<replicated> -> compute_l2

projB: ext_driver -> ext_kernel

test_scheduler_graph_multiple_separate(tmp_path, testdir, config, frontend)

Tests combining two scheduler graphs, where that an individual sub-branch is pruned in the driver schedule, while IPA meta-info is still injected to create a seamless jump between two distinct schedules for projA and projB

projA: driverB -> kernelB -> compute_l1<replicated> -> compute_l2

<ext_driver>

projB: ext_driver -> ext_kernel

test_scheduler_graph_multiple_separate_enrich_fail(testdir, config, frontend, strict, tmp_path)

Tests that explicit enrichment in “strict” mode will fail because it can’t find ext_driver

projA: driverB -> kernelB -> compute_l1<replicated> -> compute_l2

<ext_driver>

projB: ext_driver -> ext_kernelfail

test_scheduler_module_dependency(testdir, config, frontend, tmp_path)

Ensure dependency chasing is done correctly, even with surboutines that do not match module names.

projA: driverC -> kernelC -> compute_l1<replicated> -> compute_l2

projC: | –> routine_one -> routine_two

test_scheduler_module_dependencies_unqualified(testdir, config, frontend, tmp_path)

Ensure dependency chasing is done correctly for unqualified module imports.

projA: driverD -> kernelD -> compute_l1<replicated> -> compute_l2

< proj_c_util_mod>

projC: | –> routine_one -> routine_two

test_scheduler_missing_files(testdir, config, frontend, strict, tmp_path)

Ensure that strict=True triggers failure if source paths are missing and that strict=False goes through gracefully.

projA: driverC -> kernelC -> compute_l1<replicated> -> compute_l2

projC: < cannot find path >

test_scheduler_dependencies_ignore(tmp_path, testdir, preprocess, frontend)

Test multi-lib transformation by applying the DependencyTransformation over two distinct projects with two distinct invocations.

projA: driverB -> kernelB -> compute_l1<replicated> -> compute_l2

projB: ext_driver -> ext_kernel

test_scheduler_cmake_planner(tmp_path, testdir, frontend)

Test the plan generation feature over a call hierarchy spanning two distinctive projects.

projA: driverB -> kernelB -> compute_l1<replicated> -> compute_l2

projB: ext_driver -> ext_kernel

test_scheduler_item_dependencies(testdir, tmp_path)

Make sure children are correct and unique for items

fixture_loki_69_dir(testdir)

Fixture to write test file for LOKI-69 test.

test_scheduler_loki_69(loki_69_dir, tmp_path)

Test compliance of scheduler with edge cases reported in LOKI-69.

test_scheduler_scopes(tmp_path, testdir, config, frontend)

Test discovery with import renames and duplicate names in separate scopes

driver —-> kernel1_mod#kernel —-> kernel1_impl#kernel_impl

+——–> kernel2_mod#kernel —-> kernel2_impl#kernel_impl

test_scheduler_typebound(tmp_path, testdir, config, frontend, proj_typebound_dependencies)

Test correct dependency chasing for typebound procedure calls.

projTypeBound: driver -> some_type%other_routine -> other_routine -> some_type%routine1 -> routine1
| | | | | | |
| | | | | +- routine <- some_type%routine2 <-+ +———+
| | | | | | |
| | | | +–> some_type%some_routine -> some_routine -> some_type%routine -> module_routine <-+
| | +——> header_type%member_routine -> header_member_routine
| +——–> header_type%routine -> header_type%routine_real -> header_routine_real
| |
| +—> header_type%routine_integer -> routine_integer
+———->other_type%member -> other_member -> header_member_routine <–+

+————>other_type%var%%member_routine -> header_type%member_routine –+

test_scheduler_typebound_ignore(tmp_path, testdir, config, frontend, proj_typebound_dependencies)

Test correct dependency chasing for typebound procedure calls with ignore working for typebound procedures correctly.

projTypeBound: driver -> some_type%other_routine -> other_routine -> some_type%routine1 -> routine1
| | | | | |
| | | | +- routine <- some_type%routine2 <-+ +———+
| | | | | |
| | | +–> some_type%some_routine -> some_routine -> some_type%routine -> module_routine <-+
| +——> header_type%member_routine -> header_member_routine
+——–> header_type%routine -> header_type%routine_real -> header_routine_real
+—> header_type%routine_integer -> routine_integer

+———->other_type%member -> other_member -> header_member_routine

test_scheduler_traversal_order(tmp_path, testdir, config, frontend, use_file_graph, reverse)

Test correct traversal order for scheduler processing

test_scheduler_member_routines(tmp_path, config, frontend, use_file_graph, reverse)

Make sure that transformation processing works also for contained member routines

Notably, this does currently _NOT_ work and this test is tmp_path to document that fact and serve as the test base for when this has been corrected.

test_scheduler_nested_type_enrichment(tmp_path, frontend, config)

Make sure that enrichment works correctly for nested types across multiple source files

test_scheduler_interface_inline_call(tmp_path, testdir, config, frontend)

Test that inline function calls declared via an explicit interface are added as dependencies.

test_scheduler_interface_dependencies(tmp_path, frontend, config)

Ensure that interfaces are treated as intermediate nodes and incur dependencies on the actual procedures

test_scheduler_item_successors(testdir, config, frontend, tmp_path)

Test that scheduler.item_successors always returns the original item.

test_scheduler_successors(tmp_path, config, trafo_item_filter)
test_scheduler_typebound_inline_call(tmp_path, config, full_parse)
test_scheduler_cycle(tmp_path, config, full_parse)
test_scheduler_unqualified_imports(config)

Test that only qualified imports are added as children.

test_scheduler_depths(testdir, config, frontend, tmp_path)
test_scheduler_disable_wildcard(testdir, config, tmp_path)
test_transformation_config(config)

Test the correct instantiation of Transformation objecst from config

test_transformation_config_external_with_dimension(testdir, config)

Test instantiation of Transformation from config with Dimension argument.

test_scheduler_config_match_item_keys(item_name, keys, use_pattern_matching, match_item_parents, expected)
test_scheduler_filter_items_file_graph(tmp_path, frontend, config)

Ensure that the items list given to a transformation in a file graph traversal is filtered to include only used items

test_scheduler_frontend_args(tmp_path, frontend, frontend_args, defines, preprocess, has_cpp_directives, additional_dependencies, config)

Test overwriting frontend options via Scheduler config

test_scheduler_frontend_overwrite(tmp_path, config)

Test the use of a different frontend via Scheduler config

test_scheduler_pipeline_simple(testdir, config, frontend, tmp_path)

Test processing a Pipeline over a simple call-tree.

projA: driverA -> kernelA -> compute_l1 -> compute_l2

–> another_l1 -> another_l2
test_pipeline_config_compose(config)

Test the correct instantiation of a custom Pipeline object from config.

test_scheduler_indirect_import(frontend, tmp_path, enable_imports, import_level)
test_scheduler_ignore_external_item(frontend, tmp_path)