Change-Id: I869e998e683506575ecb7662c28ce49fc0b3ee00

Change-Id: I7219a82b968f747e939a4a657a0ec90f8123930d

Change-Id: I76b10f096147335cf3c827c78effee4d85feb7d0

Change-Id: Icb61b3b9e63cf77411b83e78b6ebc2d056346967

Change-Id: Ie37b246d07628b9d457ebda91990e1eac33ae50e

Change-Id: If044722deba74f573ff35140da7a2b1804f29448

Quiggeldy sometimes failes abnormally when using the async mode — we
don't care that much as this operation mode isn't actively used at the
moment (bug #3973).

Change-Id: I51ac7fbf1feed1c8eefccb24b813ef542b7bd145

* check for joinable() and set runnable flag to false beforehand
* fixes terminate without active exception in timeoutnotification-
  induced throws in upper layers

Change-Id: Ia1d760e871ad3bdce490d6df5aa6f79dadd2ffa0

* terminate is called by ~thread()
  triggered by throw in check_compatibility without join of thread
* construct thread only after check_compatibility()

Change-Id: I0a7b67fbc2fdf14cb4cb5071a7e810e4a4c9c786

Change-Id: If380fb6ebb7556504e9d39a1586469e43df16157

Change-Id: Idc92fe7d9ce9bffb024c969bbd93b7d20a35ebf7

Change-Id: Iaec0dedbd035242d5f330c1fa0c81b4636efdc88

Change-Id: I82b4b225d1d8c557375c4c57857a8ae7b55fdb5e

Depends-On: 15712, 15713, 15714, 15715
Change-Id: I585db0a22b1b938b66b16ec8c84b92fe841fb15b

Change-Id: I2251d707f400390666b6fe835c29b0269d713bef

Oliver Breitwieser authored 3 years ago and Eric Müller committed 3 years ago

Currently there is a segfault when using hxtorch if RCF deinits.

This can be workarounded by not performing a full RCF-deinit if
QuiggeldyConnection goes out of scope, but should be investigated
further.

Change-Id: I88b3cbb8277546361752c268a45f41156bb8d482

Oliver Breitwieser authored 3 years ago and Eric Müller committed 3 years ago

Change-Id: I93d90204df781f2089e8faf70141c0f8b6a509b5

Oliver Breitwieser authored 3 years ago and Eric Müller committed 3 years ago

Change-Id: I777852523e42f2ae6042c523e682db6ecb244f12

Oliver Breitwieser authored 3 years ago and Eric Müller committed 3 years ago

Change-Id: I958945d57f9bac9529a5b9f4132fab8d885ef9dc

Oliver Breitwieser authored 3 years ago and Eric Müller committed 3 years ago

Depends-On: 14214
Change-Id: I52ee7d4db9351956f7dfd2f150ed2abab227ecc2

Oliver Breitwieser authored 3 years ago and Eric Müller committed 3 years ago

Depends-On: 14179
Change-Id: Ia25a382f18afc8aba91ae5e9153a3054c99489d8

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

Depends-On: 14147
Change-Id: I07684b97dd75522ace5855f5cbd6d80bb2ba3260

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

Munge raises an error if credentials are decoded several times. However,
we usually encode user data that is set once per call. For submitting
reinit programs we perform several actions over the same client. Hence,
we need to ignore this kind of error.
If all calls set user data anew, we could drop this. Realistically this
should be fine though.

Change-Id: I9b1376107cb522bcd90d192b6e8dd562561f6ad7

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

Change-Id: I674cbdc62431c7986b2326e37300a16d6ac6e156

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

* Add FPGA timeout nofication excplicitly to ConnectionParameters for
  immediate access.
* Introduces QuiggeldyWorker<T>::check_for_timeout(response) checking
  for responses that are timeouts; triggers a reconnect if encountered.

Change-Id: Ib290317b6fe1c36a44e24600a535964ef0ebf52f

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

Change-Id: I92b6a3ab17de0ad0dbbaff7dfe1dfd0ef3eeed64

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

Add tool to query remote version information from quiggeldy connection
via RCF-interface.

```
Query remote version information of running quiggeldy instance.

Allowed options:
  -h [ --help ]         produce help message
  -i [ --ip ] arg       IP to connect to
  -p [ --port ] arg     Port to connect to
```

Depends-On: 13092
Change-Id: I68ab5f2e6e6abf21ceaec27065a75c0d2925d0e7

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

For non-munge use, display the user supplied username. For munge usage,
convert uid to string.

Makes for much more informative log messages especially when not using
munge.

Unfortunately, despite avoiding copying, using strings as user-id seems
to cause memory corruption after about five minutes of intense usage.
Investigation pending.

Change-Id: If4626e078a41ce8d4b7059492fea1137db94bfa4

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

The error messages resulting from misconfigured munge-support are rather
cryptic. Therefore, we have the client ask the server if it expects
munge to be used and then:

* deactivate munge support on the client if server does not use it.
* throw an error if client has no munge support but server expects it.

Change-Id: I833b141c6658c6db95a0d06e924a4467c260935c

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

Right now, get_unique_identifier() requires an active connection because
it is not implemented as a static method that checks the environment
rather than an existing connection.

Because hxtorch depends on this information during set up, we need to
ensure the connection is established.

Depends-On: 13932

Change-Id: I167c5ced640921cde62d6d40effe6f68b04b8f51

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

This change allows clients to specify a custom username when using
quiggeldy: QUIGGELDY_USER_NO_MUNGE

As the name indicates, this will disable munge-based authentication to
quiggeldy and hence requires that munge support is disabled on the
running quiggeldy.

This is useful if several users are emulated by a single unix user, as
was the case at the NICE2021 demo.

Depends-On: 13915

Change-Id: Iadacd1fbf25a57d3c3745bfb26140cd44291181a

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

* Add templated hxcomm::ReinitStackEntry
  * Allows registration of UTMessage-vectors to be executed after
    relinquishing control of remote hardware resources.
  * Is a no-op for all connections but QuiggeldyClient as only quiggeldy
    supports this mode of operation.
  * In Python, entries should be .pop()-ed from the stack because we
    cannot rely on the garbage collector to call the destructor in time.

* Add specialized hxcomm::vx::RegisterReinit

* Add helpers to wrap RegisterReinit in upper layers that can be
  constructed from Handles

Dependendcies:

Real dependency:
Depends-On: 11390,14013

We should also ensure the tests get built:
Depends-On: 11391

Change-Id: I33d640aa569aa4d10802469542bc41f40018ed5b

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

Quiggeldy is planned to be auto-deployed in a nightly fashion. Because
from time to time there are bugfixes/new features added, we need a way
to track in which state quiggeldy was compiled.

This is meant for expert users to ensure a specific version of quiggeldy
is deployed upon debugging.

At a later time, we will add a versioned interface, preventing users
from connecting when versions mismatch.

* Add `-v`/`--version` switch to quiggeldy that prints out with which
  repository at what commit-state it was compiled.
* Add `get_version_string`-method to RCF-interface to allow for remote
  querying of build information.

Depends-On: 13086
Change-Id: I9734f1d1d0214c4953f97e3835534f0e4d93520c

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

* Add `wriggeldy`, a wrapper to run processeses with a local quiggeldy.

* Wriggeldy is meant for debugging purposes to see if adding quiggeldy
  inbetween hardware and executing script has any detrimental effects
  (hint: it should not).

* Additionally, also add `wrap_with_quiggeldy`, a more verbose symlink
  to wriggeldy (that noone wants to type).

Change-Id: I0996832348a3a3132f9bb9adc84a96a53c6a27b5

Change-Id: I1c8b6aa70aea911a0351372ca540c3fa58eda8f2

Oliver Breitwieser authored 4 years ago and Eric Müller committed 3 years ago

(Adapted from c/9752. Change moved from fisch to hxcomm.)

* QuiggeldyClient
  * Implemented as yet another connection thanks to the new architecture
    rework.
  * Since quiggeldy is only to be used for "complete" PlaybackPrograms
    sent to the FPGA, the corresponding Stream instance only supports a
    `submit_blocking`-method that - as the name suggests - blocks until
    results are ready.
  * A correspdonding `submit_future`-method that returns a future to the
    results is planned (but currently only in proof-of-concept stage).

* Provide serialization to SF via translation function that uses
  PortableBinaryArchives from cereal.

* Generic versions of `Quiggeldy{Connection,Server,Worker}` implemented in
  `hxcomm/common/{,,detail/}quiggeldy_{rcf,common,connection,server}.*`
  * then specialized in
    `hxcomm/vx/{,,detail/}quiggeldy_{rcf,common,connection,server}.*`
    -> users only have to include specialized-headers (see tests)
  * allows straight-forward port to different sets of
    connectionparamters

* `hxcomm/{common,vx}/quiggeldy_rcf.{h,tcc}`
  * have single RCF-interface for all types of connections per
    architecture
  * SF serialization via cereal

* `hxcomm/common/quiggeldy_common.{h,tcc}`
  * needed munge auth routines for connection (client) and server
    (defined inline, should be changed if hxcomm ceases to be
    header-only)

* Seperate client and server implementation:
  * client-side implementation in `hxcomm/{common,vx}/quiggeldy_connection.{h,tcc}`
  * if not given, client-side infers remote connection parameters from
    `QUIGGELDY_{IP,PORT}` environment variables
  * server-side implemtation in
    `hxcomm/{common,vx}/detail/quiggeldy_server.{h,tcc}`

* automatic detection checks for "QUIGGELDY_ENABLED" defined in
  environment to be a non-zero integer

* add simple mock-test verifying that quiggeldy can be started and
  connected to

* add StreamRC-class that implements a stream interface with Reduced
  Capabilities for Remote exeCution

NOTE:
* slurm allocation is not tested yet!

Depends-On: 12362

Change-Id: I7600b3925c56597aedf16cdbf4147c0846a00b1b

Change-Id: I880ed4cfcda3fb8f5ddb0adea4822468fc47c5d6

Change-Id: Ifd638d035f30c0599f757ba7befd72182ba055f1

Change-Id: I55cabdbda5ede7807820efe53f9f09e4410ab4f6

Change-Id: Ie56ff04878cdd0c79bece3292b831988d1b8d409