Skip to content

Commit

Permalink
Initial open-source release
Browse files Browse the repository at this point in the history
  • Loading branch information
matthewr-xilinx committed Jan 31, 2023
0 parents commit c5d80cb
Show file tree
Hide file tree
Showing 604 changed files with 155,560 additions and 0 deletions.
26 changes: 26 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
The majority of this project is MIT licensed.

The exception is libpcap, which is BSD licensed.
Please see src/libpcap-1.5.3/LICENSE for libpcap's license


The associated MIT License is:

Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the “Software”),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.

37 changes: 37 additions & 0 deletions docs/000_main.dox
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
// SPDX-License-Identifier: MIT
// X-SPDX-Copyright-Text: Copyright (C) 2022, Advanced Micro Devices, Inc.

/**************************************************************************
* Main page
*************************************************************************/
/*! \mainpage Introduction

\htmlinclude disclaimer.html

SolarCapture is a toolkit for high performance packet capture and other
packet processing applications. SolarCapture includes command line
applications, utilities, and various API bindings. For more detail see the
SolarCapture User Guide.

This document describes the C bindings which are used to extend
SolarCapture and to embed it into applications.

The SolarCapture C bindings are included in the solar_capture-core package.
You should also install the solar_capture-python package which includes
tools and documentation as well as the Python language bindings.

Developers can use the SolarCapture C bindings to build high performance
packet processing applications on Linux. SolarCapture can receive and send
packets with the minimum number of CPU cycles for packet capture, network
security, NFV or other packet processing (C, C++, python) applications:

- This library can be embedded in the user’s own applications. See \ref embed.

- Users can also extend SolarCapture by providing processing nodes which can
be integrated into SolarCapture’s packet processing pipeline. See \ref extend.

The SolarCapture C bindings can be used with any network adapter that
supports SolarCapture. No AppFlex licenses are required to use the
SolarCapture C bindings.

*/
118 changes: 118 additions & 0 deletions docs/010_embed.dox
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
// SPDX-License-Identifier: MIT
// X-SPDX-Copyright-Text: Copyright (C) 2022, Advanced Micro Devices, Inc.

/**************************************************************************
* Embedding page
*************************************************************************/
/*! \page embed Embedding SolarCapture

The SolarCapture distribution includes C bindings, allowing SolarCapture to
be embedded in applications. Example code can be found at:

`/usr/share/doc/solar_capture-<version>/examples`

Applications that embed SolarCapture should include the `<solar_capture.h>`
header, and should link to the `solarcapture1` library, as shown in the sample
code. The header files can be found at:

`/usr/include/solar_capture/`

The following sections describe the main objects and concepts used in
SolarCapture. For more information please refer to the example code, and
the other documentation in this Guide.

\section embed_sessions Sessions — struct sc_session

All applications embedding SolarCapture must first instantiate a session
object. A session provides an association between SolarCapture components.

All objects in a SolarCapture session are allocated up front, and packet
processing is then initiated by calling sc_session_go(). Once packet
processing has started it is not possible add new objects to the session.

\section embed_attributes Attributes — struct sc_attr

Attributes provide a convenient way to specify options, such as the size of
buffers. For detailed information concerning SolarCapture attributed, refer
to SolarCapture Attributes on page 108.

\section embed_threads Threads — struct sc_thread

A session includes one or more threads that work together. Threads can be
used for packet capture, packet processing and other tasks. The threads in
a particular session are started and stopped as a group.

Objects that are part of the data-path are associated with a particular
thread and are only accessed by that thread. This allows SolarCapture to
operate without locks or expensive atomic operations, and helps to avoid
sharing state between CPU caches.

A thread can be bound to a particular CPU core by setting the `affinity_core`
attribute.

By default SolarCapture threads use busy-waiting. That is, they consume
CPU cycles even when they do not have any work to do. Threads can be
configured to sleep when idle by setting the `busy_wait` attribute to 0.

\section embed_virtual_interfaces Virtual Interfaces — struct sc_vi

A virtual interface (VI) receives packets from a network interface, and
passes them on to a node. The sc_stream interface is used to indicate
which packets should be steered to a particular VI.

\section embed_nodes Nodes — struct sc_node

Nodes perform processing on packets. SolarCapture includes many node types
that can be used in applications, and new node types can be implemented
using the \ref extend API.

VIs and nodes are connected in a directed acyclic graph, with node links
passing packet buffers from one node to another. The buffers that are
passed between nodes don't have to contain packets: They can contain any
sort of data or messages. Nodes can be used to inspect or modify the
packet buffers, generate new packet buffers, perform custom processing or
interact with other parts of the system.

Connections can be made between nodes in the same thread or in different
threads, provided that the threads are in the same SolarCapture session.
Connections between threads use mailboxes, which are created automatically.

Packet buffers are allocated by packet pools. Many nodes receive packet
buffers from VIs or other nodes via incoming links. It is also possible
for nodes to allocate buffers from a pool. Buffers are freed back to their
pool by forwarding them through a node link that is not connected.

A user application can consist of one or more nodes which may co-operate in
order to progressively process the received network packets.

\image html example_app_nodes.png "Example Application Nodes"
\image rtf example_app_nodes.png "Example Application Nodes"
\image latex example_app_nodes.png "Example Application Nodes"

The figure above is an example of co-operating nodes in a stock trading
environment. Captured packets are fed to a filtering node which selects
packets of interest to be forwarded to a second node for further analysis.
All other packets a fed to the disk writer node. The analysis node will
conduct further analysis on the packets such as statistics collection before
passing packets to the disk writer node.

\section embed_mailboxes Mailboxes — struct sc_mailbox

Mailboxes are used to pass packet buffers between nodes in different
threads, using an efficient lock-free mechanism. Each mailbox is paired
with another in a different thread, and packets can be passed through a
pair of mailboxes in both directions.

Mailboxes are created automatically when nodes in different threads are
connected. Applications can create mailboxes explicitly if they need more
fine-grained control.

\image html mailboxes.png "Mailboxes"
\image rtf mailboxes.png "Mailboxes"
\image latex mailboxes.png "Mailboxes"

\section embed_built_in_nodes Built-In Nodes

Many built-in nodes are available. These are documented in \ref nodes.

*/
124 changes: 124 additions & 0 deletions docs/020_extend.dox
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
// SPDX-License-Identifier: MIT
// X-SPDX-Copyright-Text: Copyright (C) 2022, Advanced Micro Devices, Inc.

/**************************************************************************
* Extending page
*************************************************************************/
/*! \page extend Extending SolarCapture

SolarCapture defines a coherent API allowing applications to be constructed
from reusable components known as nodes. The core SolarCapture functionality
can be extended by implementing new types of nodes in C. An example of how to
define a new node type can be found at:

`/usr/share/doc/solar_capture-<version>/examples/extensions_api`

Implementations of new node types should include the `<solar_capture.h>`
header, and should link to the `solarcapture1` library, as shown in the sample
code. The header files can be found at:

`/usr/include/solar_capture/`

This chapter describes the objects and concepts needed to create new nodes.
For more information please refer to the example code, and the other
documentation in this Guide.

\section extend_node_factories Node factories — struct sc_node_factory

A node factory provides an interface for instantiating new nodes. When a
node is allocated with sc_node_alloc() or similar, the nf_init_fn() is
invoked which should initialize the implementation and set the node type.
Private state for the node implementation can be stored in the nd_private
field.

The nf_init_fn() can retrieve arguments passed when allocating a node by
invoking the following functions:

- sc_node_init_get_arg_int()
- sc_node_init_get_arg_int64()
- sc_node_init_get_arg_dbl()
- sc_node_init_get_arg_str()
- sc_node_init_get_arg_obj()


\section extend_node_types Node types — struct sc_node_type

This object defines the behavior of a node via a set of
callbacks. Implementations must only instantiate objects of this type by
calling sc_node_type_alloc(). A single node type instance can be shared by
multiple node instances.

The nt_prep_fn() callback is invoked once per node just before the threads in
a session are started. The outgoing links configured by the application are
passed to this function. For nodes where the names of links can be chosen by
the application, the links array should be inspected directly. Nodes that
support links with fixed names can use the following functions to find
their links:

- sc_node_prep_get_link()
- sc_node_prep_get_link_or_free()

The nt_pkts_fn() callback is invoked when packets arrive at a node. This
callback provides the core functionality of the node. Packets provided to
this callback should be forwarded via one of the node’s outgoing links with
sc_forward() or sc_forward_list(). (Packets do not have to be forwarded
immediately).

The nt_end_of_stream_fn() callback is invoked when a node has received the
last packet. That is, nt_pkts_fn() is never invoked after
nt_end_of_stream_fn().


\section extend_node_libraries Node libraries

A node library is a shared object file that contains one or more
sc_node_factory instances. Each factory instance must be named
`<something>_sc_node_factory` so that it can be found by sc_node.

If a node library contains a single factory, it is conventional to give the
factory and the file matching names so that it is not necessary to name the
library in the call to sc_node_factory_lookup(). For example, in the
“reflect” example, the factory instance is `reflect_sc_node_factory`, and
the library is `reflect.so`. If a node library is placed in one of the
directories on the node library lookup path, then it will be found by a
call to sc_node_factory_lookup(), sc_node_alloc_named() or
sc_node_alloc_from_str().

The node library lookup path includes the following directories:

- . (The current working directory)
- Directories identified by the environment variable SC_NODE_PATH
- /usr/lib64/solar_capture/site-nodes
- /usr/lib/x86_64-linux-gnu/solar_capture/site-nodes
- /usr/lib64/solar_capture/nodes
- /usr/lib/x86_64-linux-gnu/solar_capture/nodes

\note Node factories do not have to be placed in node libraries. They can
simply be instantiated within an application that embeds SolarCapture and
be passed directly to sc_node_alloc(). Node libraries are useful when
nodes are reusable.


\section extend_user_defined_node Insert a user-defined node between capture and sc_writer

User-defined nodes can be inserted between the capture node and sc_writer
node. See the extensions_api sample code for examples included in the
solar_capture-python RPM.

The following example demonstrates how to insert a user-defined node called
‘`header_strip`’ into the solar_capture pipeline:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.sh}
# SC_NODE_PATH must include directory containing header_strip.so
export SC_NODE_PATH=/path/to/nodes
solar_capture eth4=/captures/eth4.pcap header_strip:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The following example demonstrates how to pass arguments to the
‘header_strip’ node:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.sh}
solar_capture eth4=/captures/eth4.pcap "header_strip:arg1=foo;arg2=bar"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

*/
Loading

0 comments on commit c5d80cb

Please sign in to comment.