feat: add request/response support to server ActionDispatcher

[martin-fleck-at]

What it does

Enable the server to send requests to the client and await responses, and to issue requests handled locally by server-side handlers. Complements the client-side changes in glsp-client.

• Add request() and requestUntil() to ActionDispatcher
• Intercept responses in dispatch() and doDispatch() to resolve pending requests
• Translate RejectAction responses to promise rejections
• Bypass action queue for nested requests to prevent deadlocks
• Tighten shouldForwardToClient() to use hasValidResponseId()
• Add async handleClientRequest() in DefaultGLSPServer.process()
• Add tests for request/response, deadlocks, timeouts, late responses, and dispose cleanup

Relates to eclipse-glsp/glsp#607

How to test

• Try together with feat: add bidirectional request/response protocol support glsp-client#480
• Add some request actions (selection, editor context, etc.) to a server-side handler and see that you receive the result as expected.

Follow-ups

Changelog

• This PR should be mentioned in the changelog
• This PR introduces a breaking change (if yes, provide more details below for the changelog and the migration guide)

It adds two methods to the server-side action dispatcher. We do have default implementations but if adopters have their own implementation, they will be missing those methods.

Note: Will break until the protocol changes in eclipse-glsp/glsp-client#480 are merged.

[tortmayr]

Here are my review comments after a first high level review and quick testing.

One additional issue:

We currently have no actual server side request in the example code i.e. no way to test the feature e2e or manually. We currently solely rely on unit tests.
Maybe it would be a good to exetend the workflow example with handler that actually uses the new request handling. Could be something simlple like retrieving the current editor context from the client and logging it.

[tortmayr]

Not directly related to this PR, but maybe an issue that we should track in a ticket:

This queue skipping for client actions could be problematic if the server has also registered an handler for this action kind.

In that case, the server handled actions escapes the sequential order is dispatched directly before any other queued server actions.

[martin-fleck-at]

I adapted the code now that client actions do not skip the queue automatically but instead we have a proper dispatchDirectly to dispatch actions without the queue, e.g., for progress reporting.

[martin-fleck-at]

@tortmayr I updated the PR and now we have a more explicit handling of actions:

• dispatch: Dispatches actions in order, i.e., queues them. That is true for server actions and client actions.
• dispatchDirectly: Dispatches actions immediately without queue. That is true for handler return actions or if called explicitly (e.g., progress reporting).
• interceptPendingResponse (internal): Responses to requests do not need to be re-dispatched but simply resolve the request. If there is no matching request, we queue the action.

As for the example, there really is no good place in the workflow example to do something semantically useful. However, with the upcoming MCP feature, we will need it for the tool functions and there the usage will be well-tested. Is that enough?

[tortmayr]

Following our discussion offline, I would like to see the following changes

• Revert the introduction of dispatchDirectly. Instead use the same approach as
  in the Java Server. (Actions that are dispatched/returned from within an action handler bypass the queue). AsyncLocalStorage can be used for this
• Replace custom PromiseQueue. Our current custom promise queue is a dated custom implementation that has a couple of issues in particular with error delegation/swalloing and unnecessary overheads.The current PromiseQueue should be replaced with a channel + consumer loop pattern that mirrors the Java architecture directly.
  The pattern could look like this:

class ActionChannel<T> {
    private queue: Array<{
        item: T;
        resolve: (v: void) => void;
        reject: (e: any) => void;
    }> = [];
    private notify: (() => void) | null = null;
    private stopped = false;

    push(item: T): Promise<void> {
        if (this.stopped) {
            return Promise.reject(new Error('Channel is closed'));
        }
        return new Promise((resolve, reject) => {
            this.queue.push({ item, resolve, reject });
            this.notify?.();
        });
    }

    async *consume(): AsyncGenerator<{
        item: T;
        resolve: (v: void) => void;
        reject: (e: any) => void;
    }> {
        while (!this.stopped || this.queue.length > 0) {
            while (this.queue.length > 0) {
                yield this.queue.shift()!;
            }
            if (!this.stopped) {
                await new Promise<void>(r => { this.notify = r; });
            }
        }
    }

    stop(): void {
        this.stopped = true;
        this.notify?.();
    }

    rejectPending(): void {
        for (const entry of this.queue) {
            entry.reject(new Error('Channel cleared'));
        }
        this.queue = [];
    }

    get size(): number { return this.queue.length; }
}

In addition, we should get rid of the second ephemeral queue we use for ordering responses. It is completely unnecessary instead we can just use a for-each await loop.

[tortmayr]

As discussed offline, I think this change is a significant behavioral break for adopters. Instead we should use the same approach as in the Java Server. i.e. all actions that are dispatched or returned from within an action handler are bypassing the queue and dispatched directly.

We can use an AsyncLocalStorage (node builtin) to detect these actions.