Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CpuBoundWork#CpuBoundWork(): don't spin on atomic int to acquire slot #9990

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

CpuBoundWork#CpuBoundWork(): don't spin on atomic int to acquire slot #9990

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
424e1bc
Refactor: pass CpuBoundWork& into HttpHandler#HandleRequest()
Al2Klimov Feb 7, 2024
4374c36
/v1/events: just release CpuBoundWork instead of using IoBoundWorkSlot
Al2Klimov Feb 7, 2024
46378f9
Remove unused IoBoundWorkSlot
Al2Klimov Feb 7, 2024
6d5b447
CpuBoundWork#~CpuBoundWork(): just call #Done() to deduplicate code
Al2Klimov Feb 7, 2024
b413287
[Refactor] CpuBoundWork#CpuBoundWork(): require an io_context::strand
Al2Klimov Feb 20, 2024
26ef66e
CpuBoundWork#CpuBoundWork(): don't spin on atomic int to acquire slot
Al2Klimov Feb 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 39 additions & 38 deletions lib/base/io-engine.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,60 +16,57 @@

using namespace icinga;

CpuBoundWork::CpuBoundWork(boost::asio::yield_context yc)
CpuBoundWork::CpuBoundWork(boost::asio::yield_context yc, boost::asio::io_context::strand& strand)
: m_Done(false)
{
auto& ioEngine (IoEngine::Get());
auto& sem (ioEngine.m_CpuBoundSemaphore);
std::unique_lock<std::mutex> lock (sem.Mutex);

for (;;) {
auto availableSlots (ioEngine.m_CpuBoundSemaphore.fetch_sub(1));
if (sem.FreeSlots) {
--sem.FreeSlots;
return;
}

if (availableSlots < 1) {
ioEngine.m_CpuBoundSemaphore.fetch_add(1);
IoEngine::YieldCurrentCoroutine(yc);
continue;
}
auto cv (Shared<AsioConditionVariable>::Make(ioEngine.GetIoContext()));
bool gotSlot = false;
auto pos (sem.Waiting.insert(sem.Waiting.end(), IoEngine::CpuBoundQueueItem{&strand, cv, &gotSlot}));
Comment on lines +32 to +33
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why you're using a boolean pointer here! Why not just use a simple bool type instead?


break;
}
}
lock.unlock();

CpuBoundWork::~CpuBoundWork()
{
if (!m_Done) {
IoEngine::Get().m_CpuBoundSemaphore.fetch_add(1);
}
}
try {
cv->Wait(yc);
} catch (...) {
Comment on lines +37 to +39
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are you trying to catch here? AsioConditionVariable#Wait() asynchronously waits with the non-throwing form of Asio async_wait().

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mainly catch forced_unwind.

std::unique_lock<std::mutex> lock (sem.Mutex);

void CpuBoundWork::Done()
{
if (!m_Done) {
IoEngine::Get().m_CpuBoundSemaphore.fetch_add(1);
if (gotSlot) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just use pos->GotSlot instead here and don't need to keep track of a bool type.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Items get moved out of sem.Waiting which invalidates pos. gotSlot tells me whether pos is still valid or not.

Copy link
Member

@yhabteab yhabteab Feb 22, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cppreference says this:

Adding, removing and moving the elements within the list or across several lists does not invalidate the iterators or references. An iterator is invalidated only when the corresponding element is deleted.

I would simply use a pointer to CpuBoundQueueItem for the queue instead then.

IoEngine::CpuBoundQueueItem item{&strand, cv, false};
auto pos (sem.Waiting.emplace(sem.Waiting.end(), &item));

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only when the corresponding element is deleted.

Exactly that gotSlot tells.

lock.unlock();
Done();
} else {
sem.Waiting.erase(pos);
}

m_Done = true;
throw;
}
}

IoBoundWorkSlot::IoBoundWorkSlot(boost::asio::yield_context yc)
: yc(yc)
{
IoEngine::Get().m_CpuBoundSemaphore.fetch_add(1);
}

IoBoundWorkSlot::~IoBoundWorkSlot()
void CpuBoundWork::Done()
{
auto& ioEngine (IoEngine::Get());
if (!m_Done) {
auto& sem (IoEngine::Get().m_CpuBoundSemaphore);
std::unique_lock<std::mutex> lock (sem.Mutex);

for (;;) {
auto availableSlots (ioEngine.m_CpuBoundSemaphore.fetch_sub(1));
if (sem.Waiting.empty()) {
++sem.FreeSlots;
} else {
auto next (sem.Waiting.front());

if (availableSlots < 1) {
ioEngine.m_CpuBoundSemaphore.fetch_add(1);
IoEngine::YieldCurrentCoroutine(yc);
continue;
*next.GotSlot = true;
sem.Waiting.pop_front();
boost::asio::post(*next.Strand, [cv = std::move(next.CV)]() { cv->Set(); });
}

break;
m_Done = true;
}
}

Expand All @@ -88,7 +85,11 @@ boost::asio::io_context& IoEngine::GetIoContext()
IoEngine::IoEngine() : m_IoContext(), m_KeepAlive(boost::asio::make_work_guard(m_IoContext)), m_Threads(decltype(m_Threads)::size_type(Configuration::Concurrency * 2u)), m_AlreadyExpiredTimer(m_IoContext)
{
m_AlreadyExpiredTimer.expires_at(boost::posix_time::neg_infin);
m_CpuBoundSemaphore.store(Configuration::Concurrency * 3u / 2u);

{
std::unique_lock<std::mutex> lock (m_CpuBoundSemaphore.Mutex);
m_CpuBoundSemaphore.FreeSlots = Configuration::Concurrency * 3u / 2u;
}

for (auto& thread : m_Threads) {
thread = std::thread(&IoEngine::RunEventLoop, this);
Expand Down
64 changes: 33 additions & 31 deletions lib/base/io-engine.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,22 @@
#include "base/exception.hpp"
#include "base/lazy-init.hpp"
#include "base/logger.hpp"
#include "base/shared.hpp"
#include "base/shared-object.hpp"
#include <atomic>
#include <cstdint>
#include <exception>
#include <list>
#include <memory>
#include <mutex>
#include <thread>
#include <utility>
#include <vector>
#include <stdexcept>
#include <boost/exception/all.hpp>
#include <boost/asio/deadline_timer.hpp>
#include <boost/asio/io_context.hpp>
#include <boost/asio/io_context_strand.hpp>
#include <boost/asio/spawn.hpp>

namespace icinga
Expand All @@ -30,36 +35,40 @@ namespace icinga
class CpuBoundWork
{
public:
CpuBoundWork(boost::asio::yield_context yc);
CpuBoundWork(boost::asio::yield_context yc, boost::asio::io_context::strand& strand);
CpuBoundWork(const CpuBoundWork&) = delete;
CpuBoundWork(CpuBoundWork&&) = delete;
CpuBoundWork& operator=(const CpuBoundWork&) = delete;
CpuBoundWork& operator=(CpuBoundWork&&) = delete;
~CpuBoundWork();

inline ~CpuBoundWork()
{
Done();
}

void Done();

private:
bool m_Done;
};


/**
* Scope break for CPU-bound work done in an I/O thread
* Condition variable which doesn't block I/O threads
*
* @ingroup base
*/
class IoBoundWorkSlot
class AsioConditionVariable
{
public:
IoBoundWorkSlot(boost::asio::yield_context yc);
IoBoundWorkSlot(const IoBoundWorkSlot&) = delete;
IoBoundWorkSlot(IoBoundWorkSlot&&) = delete;
IoBoundWorkSlot& operator=(const IoBoundWorkSlot&) = delete;
IoBoundWorkSlot& operator=(IoBoundWorkSlot&&) = delete;
~IoBoundWorkSlot();
AsioConditionVariable(boost::asio::io_context& io, bool init = false);

void Set();
void Clear();
void Wait(boost::asio::yield_context yc);

private:
boost::asio::yield_context yc;
boost::asio::deadline_timer m_Timer;
};

/**
Expand All @@ -70,7 +79,6 @@ class IoBoundWorkSlot
class IoEngine
{
friend CpuBoundWork;
friend IoBoundWorkSlot;

public:
IoEngine(const IoEngine&) = delete;
Expand Down Expand Up @@ -125,6 +133,13 @@ class IoEngine
}

private:
struct CpuBoundQueueItem
{
boost::asio::io_context::strand* Strand;
Shared<AsioConditionVariable>::Ptr CV;
bool* GotSlot;
};

IoEngine();

void RunEventLoop();
Expand All @@ -135,29 +150,16 @@ class IoEngine
boost::asio::executor_work_guard<boost::asio::io_context::executor_type> m_KeepAlive;
std::vector<std::thread> m_Threads;
boost::asio::deadline_timer m_AlreadyExpiredTimer;
std::atomic_int_fast32_t m_CpuBoundSemaphore;
};

class TerminateIoThread : public std::exception
{
struct {
std::mutex Mutex;
uint_fast32_t FreeSlots;
std::list<CpuBoundQueueItem> Waiting;
} m_CpuBoundSemaphore;
};

/**
* Condition variable which doesn't block I/O threads
*
* @ingroup base
*/
class AsioConditionVariable
class TerminateIoThread : public std::exception
{
public:
AsioConditionVariable(boost::asio::io_context& io, bool init = false);

void Set();
void Clear();
void Wait(boost::asio::yield_context yc);

private:
boost::asio::deadline_timer m_Timer;
};

/**
Expand Down
3 changes: 2 additions & 1 deletion lib/remote/actionshandler.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ bool ActionsHandler::HandleRequest(
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
)
{
namespace http = boost::beast::http;
Expand Down
3 changes: 2 additions & 1 deletion lib/remote/actionshandler.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ class ActionsHandler final : public HttpHandler
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
) override;
};

Expand Down
3 changes: 2 additions & 1 deletion lib/remote/configfileshandler.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ bool ConfigFilesHandler::HandleRequest(
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
)
{
namespace http = boost::beast::http;
Expand Down
3 changes: 2 additions & 1 deletion lib/remote/configfileshandler.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ class ConfigFilesHandler final : public HttpHandler
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
) override;
};

Expand Down
3 changes: 2 additions & 1 deletion lib/remote/configpackageshandler.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ bool ConfigPackagesHandler::HandleRequest(
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
)
{
namespace http = boost::beast::http;
Expand Down
3 changes: 2 additions & 1 deletion lib/remote/configpackageshandler.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ class ConfigPackagesHandler final : public HttpHandler
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
) override;

private:
Expand Down
3 changes: 2 additions & 1 deletion lib/remote/configstageshandler.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ bool ConfigStagesHandler::HandleRequest(
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
)
{
namespace http = boost::beast::http;
Expand Down
3 changes: 2 additions & 1 deletion lib/remote/configstageshandler.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ class ConfigStagesHandler final : public HttpHandler
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
) override;

private:
Expand Down
3 changes: 2 additions & 1 deletion lib/remote/consolehandler.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,8 @@ bool ConsoleHandler::HandleRequest(
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
)
{
namespace http = boost::beast::http;
Expand Down
3 changes: 2 additions & 1 deletion lib/remote/consolehandler.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ class ConsoleHandler final : public HttpHandler
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
) override;

static std::vector<String> GetAutocompletionSuggestions(const String& word, ScriptFrame& frame);
Expand Down
3 changes: 2 additions & 1 deletion lib/remote/createobjecthandler.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ bool CreateObjectHandler::HandleRequest(
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
)
{
namespace http = boost::beast::http;
Expand Down
3 changes: 2 additions & 1 deletion lib/remote/createobjecthandler.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ class CreateObjectHandler final : public HttpHandler
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
) override;
};

Expand Down
3 changes: 2 additions & 1 deletion lib/remote/deleteobjecthandler.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ bool DeleteObjectHandler::HandleRequest(
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
)
{
namespace http = boost::beast::http;
Expand Down
3 changes: 2 additions & 1 deletion lib/remote/deleteobjecthandler.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ class DeleteObjectHandler final : public HttpHandler
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
) override;
};

Expand Down
5 changes: 3 additions & 2 deletions lib/remote/eventshandler.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,8 @@ bool EventsHandler::HandleRequest(
boost::beast::http::response<boost::beast::http::string_body>& response,
const Dictionary::Ptr& params,
boost::asio::yield_context& yc,
HttpServerConnection& server
HttpServerConnection& server,
CpuBoundWork& handlingRequest
)
{
namespace asio = boost::asio;
Expand Down Expand Up @@ -105,7 +106,7 @@ bool EventsHandler::HandleRequest(
response.result(http::status::ok);
response.set(http::field::content_type, "application/json");

IoBoundWorkSlot dontLockTheIoThread (yc);
handlingRequest.Done();

http::async_write(stream, response, yc);
stream.async_flush(yc);
Expand Down
Loading
Loading