[Bug] AvatarChatbot can NOT work in K8s environment because of functional gap in wav2clip service

[lianhao]

Priority

P3-Medium

OS type

Ubuntu

Hardware type

Xeon-GNR

Installation method

• Pull docker images from hub.docker.com
• Build docker images from source
• Other

Deploy method

• Docker
• Docker Compose
• Kubernetes Helm Charts
• Kubernetes GMC
• Other

Running nodes

Single Node

What's the version?

git commit 45d5da2

Description

The current AvatarChatbot example requires the users to map a local disk directory for the wav2clip service to save the result, i.e. see the volumes section in the following docker compose file definition:

wav2lip-service:
    image: ${REGISTRY:-opea}/wav2lip:${TAG:-latest}
    container_name: wav2lip-service
    ports:
      - "7860:7860"
    ipc: host
    volumes:
      - ${PWD}:/outputs
    ... ...

The user will need to read files from the current local directory of {PWD} to get the results. This kind of requirement prohibits us to deploy the AvatarChatBot in Kubernetes(K8s) environment, where the wav2clip service could be scheduled to one of the node in the k8s cluster which the end user will never have access to.

The wav2clip service need to return the generated file content, instead of just returning the filename.

Reproduce steps

n/a

Raw log

Attachments

No response

[xiguiw]

@ctao456
Can you work on this?

[xiguiw]

Remind :
@ctao456 Please help fixing this issue.

[xiguiw]

@lianhao

No fix till now.
This is listed as a known issue. Will add this in release note.

[ctao456]

Hi @lianhao thanks for pointing this out. Understand the issue. @xiguiw thanks for reminding.
However, the OPEA animation microservice has the following defined input and output datatypes:

@register_microservice(
    name="opea_service@animation",
    service_type=ServiceType.ANIMATION,
    endpoint="/v1/animation",
    host="0.0.0.0",
    port=9066,
    input_datatype=Base64ByteStrDoc,
    output_datatype=VideoPath,
)

So there's dependencies we need to consider when changing the output datatype. Especially those associated with https://github.com/opea-project/GenAIExamples/tree/main/AvatarChatbot

Also, in the actual wav2lip api server animate realization, a cv2.VideoWriter object is used to write and release the silent .avi video, which is then fused with audio by the ffmpeg software codec, to generate the result .mp4 video
https://github.com/opea-project/GenAIComps/blob/3e559df866972631e41a495b0bea068217224f86/comps/third_parties/wav2lip/src/wav2lip_server.py#L152C1-L163C33

out.release()

    ffmpeg.output(
        ffmpeg.input(args.audio),
        ffmpeg.input("temp/result.avi"),
        args.outfile,
        strict="-2",
        crf=23,
        vcodec="libx264",
        preset="medium",
        acodec="aac",
    ).run(overwrite_output=True)

    args.audio = "None"  # IMPORTANT: Reset audio to None for the next audio request

    return {"wav2lip_result": args.outfile}

It's not possible to change/add the output datatype of the actual video content.

Please refer to the above and let us know if you have a better workaround for this application to be deployed on K8s. Thanks!

[yinghu5]

Hi @lianhao
Chun update the AvatorChatBot today ：
GenAIExamples PR #1776
GenAIComps PR #1540
could you please chekc at K8S env?