`specfem::execution::ChunkedEdgeIterator`¶

template<typename ParallelConfig, typename ViewType> class ChunkedEdgeIterator : public specfem::execution::TeamPolicy<ParallelConfig>¶

High-level chunked edge iterator for efficient parallel edge processing.

This is the main iterator class for processing mesh edges in chunks. It divides the edge set into chunks of configurable size and processes each chunk in parallel using Kokkos teams. This approach improves memory locality and load balancing.

Performance¶

The chunked approach provides several benefits:

Improved memory locality by processing related edges together
Better load balancing across teams
Reduced memory pressure from large edge sets
Configurable chunk sizes for different architectures

Usage¶

#include "execution/chunked_edge_iterator.hpp"
#include "execution/for_all.hpp"

// Define parallel configuration
using ParallelConfig =
specfem::parallel_configuration::default_chunk_edge_config<
    specfem::dimension::type::dim2, Kokkos::DefaultExecutionSpace>;

// Create edge view and initialize
constexpr int num_edges = 10000;
constexpr int num_points = 5;
Kokkos::View<specfem::mesh_entity::edge<specfem::dimension::type::dim2>*>
edges("edges", num_edges);

// Initialize edges with proper element indices and types
Kokkos::parallel_for("init_edges", num_edges, KOKKOS_LAMBDA(int i) {
    edges(i) = specfem::mesh_entity::edge<specfem::dimension::type::dim2>(i,
specfem::mesh_entity::dim2::type::top);
});

// Create storage for computation results
Kokkos::View<int*[num_points]> storage("storage", num_edges);

// Create and use chunked iterator
specfem::execution::ChunkedEdgeIterator iterator(ParallelConfig(), edges,
num_points);

specfem::execution::for_all("process_edges", iterator,
    KOKKOS_LAMBDA(const auto& index) {
        // Each thread processes one edge point
        int element_id = index.ispec;
        int point_id = index.ipoint;

        // Perform computation on edge point
        Kokkos::atomic_add(&storage(element_id, point_id), 1);
    });

Kokkos::fence();

ChunkSize¶

The chunk size is determined by ParallelConfig::chunk_size. Typical values:

GPU: 32 edges per chunk
CPU: 1 edge per chunk

Template Parameters:

ParallelConfig – Configuration class defining execution parameters and chunk size
ViewType – Kokkos view type containing mesh edges

Public Types

using base_policy_type = typename base_type::base_policy_type¶: Base policy type. Evaluates to Kokkos::TeamPolicy`

using policy_index_type = typename base_type::policy_index_type¶: Policy index type. Evaluates to Kokkos::TeamPolicy::member_type

using index_type = ChunkEdgeIndex<ParallelConfig::dimension, ViewType, policy_index_type>¶: Underlying index type. This index will be passed to the closure when calling specfem::execution::for_each_level

using execution_space = typename base_type::execution_space¶: Execution space type.

using base_index_type = EdgePointIndex<ParallelConfig::dimension, int, typename base_type::execution_space>¶: Index type to be used when calling specfem::execution::for_all with this iterator.

Public Functions

inline ChunkedEdgeIterator(const ViewType edges)¶

Constructor with explicit edge view and point count.

Parameters:: edges – View of mesh edges to process

inline ChunkedEdgeIterator(const ParallelConfig, const ViewType edges)¶

Constructor with parallel configuration.

Parameters:

config – Parallel configuration (unused but required for interface compatibility)
edges – View of mesh edges to process

inline const index_type operator()(const policy_index_type &team) const¶

Team operator for chunk processing.

Creates a chunk-specific index for the given team. Each team processes a contiguous chunk of edges, improving memory locality.

Parameters:: team – Kokkos team member
Returns:: index_type Chunk index for this team

template<typename ...Args> inline ChunkedEdgeIterator &set_scratch_size(Args&&... args)¶

Set scratch memory size for teams.

Forwards scratch memory configuration to the underlying team policy.

Template Parameters:: Args – Variadic template for scratch size arguments
Parameters:: args – Arguments to forward to team policy
Returns:: ChunkedEdgeIterator& Reference to this iterator for chaining

Public Static Attributes

static constexpr PolicyType policy_type = PolicyType::KokkosPolicy ¶: Indicates this is a Kokkos policy.

static constexpr bool is_top_level_policy = true¶: Indicates this is a top-level policy.

`specfem::execution::ChunkEdgeIndex`¶

template<specfem::dimension::type DimensionTag, typename ViewType, typename KokkosIndexType> class ChunkEdgeIndex¶

Chunk-level index for managing edge processing within a team.

This class serves as an intermediate index type that manages a chunk of edges assigned to a Kokkos team. It provides access to both the chunk-specific index and the team-level iterator for processing edge points within the chunk.

Architecture¶

ChunkEdgeIndex acts as a bridge between:

High-level ChunkedEdgeIterator (manages teams and chunks)
Team-level ChunkEdgeIterator (processes edges within a chunk)
Low-level EdgePointIndex (represents individual edge points)

Usage¶

// Typically used internally within the chunked iterator hierarchy
// Not directly instantiated by user code

// Example of accessing chunk index in nested iteration:
specfem::execution::for_all("process_chunks", chunked_iterator,
    KOKKOS_LAMBDA(const auto& chunk_index) {
        // chunk_index is of type ChunkEdgeIndex
        auto team_iterator = chunk_index.get_iterator();
        auto policy_idx = chunk_index.get_policy_index();

        // Use team_iterator for further nested iteration
        // within this chunk of edges
    });

Responsibilities¶

Maintains reference to the chunk’s edge subset view
Stores the Kokkos team member for parallel execution
Provides access to team-level iterator for edge processing
Bridges between chunk-level and point-level operations

Template Parameters:

DimensionTag – Spatial dimension (2D or 3D)
ViewType – Kokkos view type containing the chunk of mesh edges
KokkosIndexType – Type of the underlying Kokkos policy index (team member)

Public Functions

inline constexpr const KokkosIndexType &get_policy_index() const¶

Get the Kokkos policy index (team member) for this chunk.

Returns:: const KokkosIndexType& Reference to the Kokkos team member that is responsible for processing this chunk

inline const index_type get_index() const¶

Get the chunk-specific index.

Returns:: const index_type& Reference to the chunk index containing metadata about this specific chunk of edges

inline const iterator_type &get_iterator() const¶

Get the team-level iterator for processing edges in this chunk.

Returns:: const iterator_type& Reference to the ChunkEdgeIterator that can be used to iterate over individual edge points within this chunk

inline ChunkEdgeIndex(const ViewType &edges, const KokkosIndexType &kokkos_index)¶

Constructor for ChunkEdgeIndex.

Parameters:

edges – View of mesh edges for this specific chunk (subset of total edges)
num_points – Number of GLL points per edge
kokkos_index – Kokkos team member responsible for this chunk

`specfem::execution::ChunkEdgeIterator`¶

template<specfem::dimension::type DimensionTag, typename ViewType, typename TeamMemberType> class ChunkEdgeIterator : public specfem::execution::TeamThreadRangePolicy<TeamMemberType, int>¶

Team-level iterator for processing edge points within a chunk.

This iterator operates at the team level, distributing edge points among team members for parallel processing. It computes local element coordinates for each edge point based on the edge type and orientation.

EdgeMapping¶

Edge points are mapped to element coordinates based on edge type:

bottom: (iz=0, ix=ipoint)
top: (iz=num_points-1, ix=num_points-1-ipoint)
left: (iz=ipoint, ix=0)
right: (iz=num_points-1-ipoint, ix=num_points-1)

Usage¶

// Used internally by ChunkedEdgeIterator, not typically instantiated
directly
// Access through the chunked iterator's operator()

Template Parameters:

DimensionTag – Spatial dimension (2D or 3D)
ViewType – Kokkos view type containing mesh edges
TeamMemberType – Kokkos team member type

Public Functions

inline const index_type operator()(const policy_index_type &i) const¶

Convert linear index to edge point index.

Maps a linear index to specific edge and point coordinates, handling edge orientation and element mapping.

Parameters:: i – Linear index within the thread range
Returns:: index_type EdgePointIndex for the specified linear index

inline ChunkEdgeIterator(const TeamMemberType &team_member, const ViewType &edges_)¶

Constructor for team-level edge iterator.

Parameters:

team_member – Kokkos team member for parallel execution
edges – View of mesh edges to process
npoints – Number of GLL points per edge

Public Members

const int nedges¶: Total number of edges in this chunk.

Public Static Attributes

static constexpr PolicyType policy_type¶: Indicates this is a Kokkos policy.

static constexpr bool is_top_level_policy¶: Indicates this is not a top-level policy.

`specfem::execution::EdgePointIndex`¶

template<specfem::dimension::type DimensionTag, typename KokkosIndexType, typename ExecutionSpace> class EdgePointIndex¶

Index type representing a single point on a mesh edge.

This class encapsulates the coordinates and properties of a single Gauss-Lobatto-Legendre (GLL) point located on a mesh edge. It provides access to both the local element coordinates and the position along the edge.

Usage¶

// Typically used within chunked edge iterator lambdas
specfem::execution::for_all("process_edges", iterator,
    KOKKOS_LAMBDA(const auto & iterator_index) {
      const auto index = iterator_index.get_index();
        int element_id = index.ispec;    // Element containing this edge point
        int z_coord = index.iz;          // Local z-coordinate within element
        int x_coord = index.ix;          // Local x-coordinate within element
        int point_pos = index.ipoint;    // Position along edge (0 to num_points-1)

        // Process the edge point...
    });

Template Parameters:

DimensionTag – Spatial dimension (2D or 3D)
KokkosIndexType – Type of the underlying Kokkos policy index
ExecutionSpace – Kokkos execution space for parallel operations

Public Types

using iterator_type = VoidIterator<ExecutionSpace>¶: Iterator type used to iterate over GLL points within this index. VoidIterator is used when the index refers to a single GLL point.

Public Functions

inline constexpr const KokkosIndexType get_policy_index() const¶

Get the policy index that defined this point index.

Returns:: const KokkosIndexType The policy index that defined this point index.

inline constexpr const index_type &get_index() const¶

Get a reference to this index.

Returns:: const index_type& Reference to this EdgePointIndex

inline EdgePointIndex(const specfem::point::edge_index<DimensionTag> &index_, const int iedge, const KokkosIndexType &kokkos_index)¶

Constructor for EdgePointIndex.

Parameters:

index – Local element coordinates of the edge point
iedge – Position of edge within the chunk
kokkos_index – Underlying Kokkos policy index

inline constexpr const iterator_type get_iterator() const¶

Get iterator for this single point.

Returns:: const iterator_type VoidIterator since this represents a single point

specfem::execution::ChunkedEdgeIterator¶

Performance¶

Usage¶

ChunkSize¶

specfem::execution::ChunkEdgeIndex¶

Architecture¶

Usage¶

Responsibilities¶

specfem::execution::ChunkEdgeIterator¶

EdgeMapping¶

Usage¶

specfem::execution::EdgePointIndex¶

Usage¶

`specfem::execution::ChunkedEdgeIterator`¶

`specfem::execution::ChunkEdgeIndex`¶

`specfem::execution::ChunkEdgeIterator`¶

`specfem::execution::EdgePointIndex`¶