Change-Id: I51a1bf04e2166a2dfbb2b489dbeb0d54baf0cab1

The partition for hardware runs was changed. Do no longer use partition
to decide if hw tests should be executed but other env var.

Change-Id: I9331007ec4f1aaf7200da04fe887732a096f9a3c

Eric Müller authored 1 year ago and Philipp Spilger committed 1 year ago

Change-Id: I0b21cf03bb1b397f273ca92a8b67feb8df0a70aa

* Genpybind macros not in code, SYMBOL_VISIBLE missing

Change-Id: I9c52a6861407dce9e5a218cb7a5a4878a0be39e7

We probably don't want to build EXTOLL stuff on remote (e.g. EBRAINS)
infrastructure that connects via quiggeldy anyways.

Change-Id: I68ac5044df6bd7250686ad0e924fd36c69599737

We probably don't want to build HostARQ stuff on remote (e.g. EBRAINS)
infrastructure that connects via quiggeldy anyways.

Depends-On: c/20318
Change-Id: I999496913993a662bb93dfad9754695f9da41d80

* dynamically extract downstream dependencies for repo state

Change-Id: If98f68dce3bddfab7087b2a6332b62019f766b6a

Change-Id: I3c0578398b16f0cd83c981e09ea91c4595a6f390

sborduka authored 3 years ago and Philipp Spilger committed 2 years ago

Depends-On: 17913,17759,18733,19543,19544
Change-Id: I15656b7d1da952890d01cf4be15919c7553e16e7

Change-Id: Ia4122660808b38af9eaa7c9b77513f2a8bf6db10

* uses waf to extract information

Change-Id: Ibc2fce78ea76e0a5c3812cc7fe284a641f7e3662

* performance increase 160MB/s -> 240MB/s for random messages
* easier debugability because of missing context switches

Change-Id: If45580701c647500b05ee38b86261949161e76e1

* previously we redefined the variable a second time with another
  value (empty)
* now we redefine it to the same value and therefore don't trigger
  a warning

Change-Id: Icb13e3ea8275afae2b22aa2ee6c2d03534c4487a

Depends-On: 15030
Change-Id: Icb91e532a5700ab1af650735a852cf9a03c46bba

Change-Id: Id13cfd1830b744d07e7c62a5cbaf1d967843bc3f

Change-Id: I869e998e683506575ecb7662c28ce49fc0b3ee00

Change-Id: I7219a82b968f747e939a4a657a0ec90f8123930d

Change-Id: I2251d707f400390666b6fe835c29b0269d713bef

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

Add tool to query remote version information from quiggeldy connection
via RCF-interface.

```
Query remote version information of running quiggeldy instance.

Allowed options:
  -h [ --help ]         produce help message
  -i [ --ip ] arg       IP to connect to
  -p [ --port ] arg     Port to connect to
```

Depends-On: 13092
Change-Id: I68ab5f2e6e6abf21ceaec27065a75c0d2925d0e7

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

Quiggeldy is planned to be auto-deployed in a nightly fashion. Because
from time to time there are bugfixes/new features added, we need a way
to track in which state quiggeldy was compiled.

This is meant for expert users to ensure a specific version of quiggeldy
is deployed upon debugging.

At a later time, we will add a versioned interface, preventing users
from connecting when versions mismatch.

* Add `-v`/`--version` switch to quiggeldy that prints out with which
  repository at what commit-state it was compiled.
* Add `get_version_string`-method to RCF-interface to allow for remote
  querying of build information.

Depends-On: 13086
Change-Id: I9734f1d1d0214c4953f97e3835534f0e4d93520c

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

* Add `wriggeldy`, a wrapper to run processeses with a local quiggeldy.

* Wriggeldy is meant for debugging purposes to see if adding quiggeldy
  inbetween hardware and executing script has any detrimental effects
  (hint: it should not).

* Additionally, also add `wrap_with_quiggeldy`, a more verbose symlink
  to wriggeldy (that noone wants to type).

Change-Id: I0996832348a3a3132f9bb9adc84a96a53c6a27b5

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

(Adapted from c/9752. Change moved from fisch to hxcomm.)

* QuiggeldyClient
  * Implemented as yet another connection thanks to the new architecture
    rework.
  * Since quiggeldy is only to be used for "complete" PlaybackPrograms
    sent to the FPGA, the corresponding Stream instance only supports a
    `submit_blocking`-method that - as the name suggests - blocks until
    results are ready.
  * A correspdonding `submit_future`-method that returns a future to the
    results is planned (but currently only in proof-of-concept stage).

* Provide serialization to SF via translation function that uses
  PortableBinaryArchives from cereal.

* Generic versions of `Quiggeldy{Connection,Server,Worker}` implemented in
  `hxcomm/common/{,,detail/}quiggeldy_{rcf,common,connection,server}.*`
  * then specialized in
    `hxcomm/vx/{,,detail/}quiggeldy_{rcf,common,connection,server}.*`
    -> users only have to include specialized-headers (see tests)
  * allows straight-forward port to different sets of
    connectionparamters

* `hxcomm/{common,vx}/quiggeldy_rcf.{h,tcc}`
  * have single RCF-interface for all types of connections per
    architecture
  * SF serialization via cereal

* `hxcomm/common/quiggeldy_common.{h,tcc}`
  * needed munge auth routines for connection (client) and server
    (defined inline, should be changed if hxcomm ceases to be
    header-only)

* Seperate client and server implementation:
  * client-side implementation in `hxcomm/{common,vx}/quiggeldy_connection.{h,tcc}`
  * if not given, client-side infers remote connection parameters from
    `QUIGGELDY_{IP,PORT}` environment variables
  * server-side implemtation in
    `hxcomm/{common,vx}/detail/quiggeldy_server.{h,tcc}`

* automatic detection checks for "QUIGGELDY_ENABLED" defined in
  environment to be a non-zero integer

* add simple mock-test verifying that quiggeldy can be started and
  connected to

* add StreamRC-class that implements a stream interface with Reduced
  Capabilities for Remote exeCution

NOTE:
* slurm allocation is not tested yet!

Depends-On: 12362

Change-Id: I7600b3925c56597aedf16cdbf4147c0846a00b1b

Change-Id: I880ed4cfcda3fb8f5ddb0adea4822468fc47c5d6

Change-Id: I55cabdbda5ede7807820efe53f9f09e4410ab4f6

Change-Id: I06221708e9a6ad77ef233a37be5d534a60dbb364

Also modifies target name… not sure if okay ;).

Depends-On: 14548
Change-Id: I074b12a76b529f820cbff9eb2f97b5ae9eb14393

Change-Id: Ib37a8c589209f0068f56a8f9b2e22149093032f4

Yannik Stradmann authored 3 years ago and Yannik Stradmann committed 3 years ago

The default timeout is too tight for test execution
on loaded ASIC machines.

Change-Id: I1de3714d845feb8bfb6a113152877e74faaeff11

Depends-On: 12902
Change-Id: I4dcc82641d99befb9483af17fc32b8edaf154f65

Yannik Stradmann authored 4 years ago and Philipp Spilger committed 4 years ago

* Full software connection which responds on reads with zero and shall ignore all other messages
* Adjustable ns_per_message can be used for performance testing of higher software layers
** defaults to 125M Messages/s, matching the 8 GBit/s FPGA-chip links for our parameterization
   for HX

Change-Id: I845be742f853786aaf5bdedc41db7a7120fc8e95

Change-Id: Id474f222c65207402e0676106ca32496c22a3929

* replace tbb::concurrent_queue by std::vector and mutex
  because the concurrent queue does not support thread-safe swap
* allows fast acquisition of _all_ messages from the connection
* in the typical use case of sending, waiting until halt and receiving
  this leads to less time spend with exclusive access to the connection
  and therefore more possible parallelism
* remove single-message receive() and try_receive() interface

Depends-On: 11113
Change-Id: Ifd6cebfcea3b9bc56699aaf9db741579ca074a81

* For the ARQConnection the hwdb is queried for unique identifier bits

Depends-On: 11708,11930,11946
Change-Id: I39ea8ce989534bb410094a5eaf37d68e3eb438e6

Change-Id: I5765e4204b458a1dce017564e461f67600b05a85

* use logger instead of direct cout in throughputtests

Change-Id: I343e55338499c827e8afab79c7ffd687413e8bdb

* use get_connection_from_env followed by a visit
* remove unused dispatch headers

Change-Id: I4b69d959443e4d1a6a57284ae8e96ffad69a6519

* A connection logs time of encode, decode, commit and
  execute operations
* Add logging to execute_messages, which logs in/out message count
  and time expenditure as difference of the time information before
  and after execution
* change default loglevel from info to warning in order to be able to have
  the time info print in execute_messages on info loglevel without spamming
  the normal user

Depends-On: 11111
Change-Id: I17835a9ae7f089da72a9004a18b967c117b6132d

Oliver Breitwieser authored 4 years ago and Philipp Spilger committed 4 years ago

In preparation of the upcoming integration of quiggeldy, some more
test-helper will be added that are also to be used in fisch tests. Hence
properly seperating helpers into different namespaces will prevent
possible clashes in the future.

Change-Id: I5786fde928674435f19323a27259faa09b936e50

* As discussed on 2020-04-07 after the Softie's meeting (ECM, YS, PSP,
  CM, OJB), we want to have a streamlined connection interfaces for all
  the possible ways we turn FPGA words into other FPGA words in
  hardware.

* C++-API: Split old Connection objects into two parts:
  * The connection object that the user can only create and destroy but
    which holds state necessary for communication with the backend (hw,
    CoSim, QuiggelyClient, etc) and allows some state to be set.
  * Introduce hxcomm/common/stream.h:
    * The Stream-object featuring a uniform interface handling how to
      feed and read words to/from the backend.
    * The interface is defined via templated `Stream`-class that has to
      be befriended in the corresponding connection implementations
    * The Stream-interface is verified at compile time via the
      Streamable helper-struct.

* Move `execute_words` from fisch as more generic
  `execute_messages`-function that is the entry point for all upper
  layers when communcating with backends.
  * Overloading is possible by fully or partially-specializing the
    detail::ExecutorMessages-helper struct's operator() method (not
    needed once partial function specialization arrives, so never)
  * default implementation in common namespace by adding explicit
    `send_halt_message_type` to ConnectionParameter template argument

* Rework hxcomm/common/{arq,sim}connection.{h,tcc} to use the new Stream
  interface.

* Update all tests to use streams for interacting with the hardware.

* Rename init_parameters_t -> init_parameters_type to be consistent
  (they will be used by quiggeldy but that change is not merged -> no
  API break)

* Provide get_connection_from_env-function that returns a connection
  variant over all possible connections (and is used in pyhxcomm, see
  below).

* Introduce seperate `pyhxcomm`-namespace that is private to hxcomm and
  only to be accessed from Python. It adds convenience context-wrappers
  for connections ensuring connections get closed upon exiting contexts
  in Python
  * pyhxcomm::Managed<Connection> provides enter/exit-methods
  * enter() returns pyhxcom::Handle<Connection> holding the connection
  * if Handle<Connection> gets destroyed, connection gets freed (i.e.
    disconnected)
  * Managed<Connection> also holds a weak-ptr to the returned
    ConnectionHandle object and disconnects the Handle upon exit()-ing
  * Expose union over all Connections as `pyhxcomm_vx.ConnectionHandle`
    that can be used in type annotations in Python.
  * Expose special context manager `pyhxcomm_vx.ManagedConnection` that
    automatically creates a new connection from the environment and
    returns `ConnectionHandle` (which is a concrete `<Connection>Handle`
    in python rather than the cumbersome variant type).

* Introduce hxcomm::visit_connection that allows for applying operations
  on connections wrapped in Handle-objects such as variants for hxcomm
  and pyhxcomm::ConnectionHandles. It is used in
  hxcomm::execute_messages as well as stadls::run-pybind11 bindings.
  * Functions templated over connections can be easily specialized for
    all wrapped connections via:
    * `ConnectionIsPlainGuard<Connection> = 0`
    * `ConnectionIsWrappedGuard<Connection> = 0`
  * Another possibility is to perform compile time checks using
    `ConnectionIsPlain<Connection>::value` (see `run`-implementation in
    haldls for example).

* Add `static char const name[]` attributes for all connection for more
  informative logging.

Depends-On: 10346,10335,10703,10978,11027

Change-Id: I3ff901db53a5cb94a1faa50b18dc36786246f389

Change-Id: I3a003e27bddb3633b7c79ce71db23b2041c3dc22