Introduce Plugins (#13836)

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Signed-off-by: -LAN- <laipz8200@outlook.com>
Signed-off-by: xhe <xw897002528@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: takatost <takatost@gmail.com>
Co-authored-by: kurokobo <kuro664@gmail.com>
Co-authored-by: Novice Lee <novicelee@NoviPro.local>
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
Co-authored-by: AkaraChen <akarachen@outlook.com>
Co-authored-by: Yi <yxiaoisme@gmail.com>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: twwu <twwu@dify.ai>
Co-authored-by: Hiroshi Fujita <fujita-h@users.noreply.github.com>
Co-authored-by: AkaraChen <85140972+AkaraChen@users.noreply.github.com>
Co-authored-by: NFish <douxc512@gmail.com>
Co-authored-by: Wu Tianwei <30284043+WTW0313@users.noreply.github.com>
Co-authored-by: 非法操作 <hjlarry@163.com>
Co-authored-by: Novice <857526207@qq.com>
Co-authored-by: Hiroki Nagai <82458324+nagaihiroki-git@users.noreply.github.com>
Co-authored-by: Gen Sato <52241300+halogen22@users.noreply.github.com>
Co-authored-by: eux <euxuuu@gmail.com>
Co-authored-by: huangzhuo1949 <167434202+huangzhuo1949@users.noreply.github.com>
Co-authored-by: huangzhuo <huangzhuo1@xiaomi.com>
Co-authored-by: lotsik <lotsik@mail.ru>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: nite-knite <nkCoding@gmail.com>
Co-authored-by: Jyong <76649700+JohnJyong@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: gakkiyomi <gakkiyomi@aliyun.com>
Co-authored-by: CN-P5 <heibai2006@gmail.com>
Co-authored-by: CN-P5 <heibai2006@qq.com>
Co-authored-by: Chuehnone <1897025+chuehnone@users.noreply.github.com>
Co-authored-by: yihong <zouzou0208@gmail.com>
Co-authored-by: Kevin9703 <51311316+Kevin9703@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Boris Feld <lothiraldan@gmail.com>
Co-authored-by: mbo <himabo@gmail.com>
Co-authored-by: mabo <mabo@aeyes.ai>
Co-authored-by: Warren Chen <warren.chen830@gmail.com>
Co-authored-by: JzoNgKVO <27049666+JzoNgKVO@users.noreply.github.com>
Co-authored-by: jiandanfeng <chenjh3@wangsu.com>
Co-authored-by: zhu-an <70234959+xhdd123321@users.noreply.github.com>
Co-authored-by: zhaoqingyu.1075 <zhaoqingyu.1075@bytedance.com>
Co-authored-by: 海狸大師 <86974027+yenslife@users.noreply.github.com>
Co-authored-by: Xu Song <xusong.vip@gmail.com>
Co-authored-by: rayshaw001 <396301947@163.com>
Co-authored-by: Ding Jiatong <dingjiatong@gmail.com>
Co-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: JasonVV <jasonwangiii@outlook.com>
Co-authored-by: le0zh <newlight@qq.com>
Co-authored-by: zhuxinliang <zhuxinliang@didiglobal.com>
Co-authored-by: k-zaku <zaku99@outlook.jp>
Co-authored-by: luckylhb90 <luckylhb90@gmail.com>
Co-authored-by: hobo.l <hobo.l@binance.com>
Co-authored-by: jiangbo721 <365065261@qq.com>
Co-authored-by: 刘江波 <jiangbo721@163.com>
Co-authored-by: Shun Miyazawa <34241526+miya@users.noreply.github.com>
Co-authored-by: EricPan <30651140+Egfly@users.noreply.github.com>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: sino <sino2322@gmail.com>
Co-authored-by: Jhvcc <37662342+Jhvcc@users.noreply.github.com>
Co-authored-by: lowell <lowell.hu@zkteco.in>
Co-authored-by: Boris Polonsky <BorisPolonsky@users.noreply.github.com>
Co-authored-by: Ademílson Tonato <ademilsonft@outlook.com>
Co-authored-by: Ademílson Tonato <ademilson.tonato@refurbed.com>
Co-authored-by: IWAI, Masaharu <iwaim.sub@gmail.com>
Co-authored-by: Yueh-Po Peng (Yabi) <94939112+y10ab1@users.noreply.github.com>
Co-authored-by: Jason <ggbbddjm@gmail.com>
Co-authored-by: Xin Zhang <sjhpzx@gmail.com>
Co-authored-by: yjc980121 <3898524+yjc980121@users.noreply.github.com>
Co-authored-by: heyszt <36215648+hieheihei@users.noreply.github.com>
Co-authored-by: Abdullah AlOsaimi <osaimiacc@gmail.com>
Co-authored-by: Abdullah AlOsaimi <189027247+osaimi@users.noreply.github.com>
Co-authored-by: Yingchun Lai <laiyingchun@apache.org>
Co-authored-by: Hash Brown <hi@xzd.me>
Co-authored-by: zuodongxu <192560071+zuodongxu@users.noreply.github.com>
Co-authored-by: Masashi Tomooka <tmokmss@users.noreply.github.com>
Co-authored-by: aplio <ryo.091219@gmail.com>
Co-authored-by: Obada Khalili <54270856+obadakhalili@users.noreply.github.com>
Co-authored-by: Nam Vu <zuzoovn@gmail.com>
Co-authored-by: Kei YAMAZAKI <1715090+kei-yamazaki@users.noreply.github.com>
Co-authored-by: TechnoHouse <13776377+deephbz@users.noreply.github.com>
Co-authored-by: Riddhimaan-Senapati <114703025+Riddhimaan-Senapati@users.noreply.github.com>
Co-authored-by: MaFee921 <31881301+2284730142@users.noreply.github.com>
Co-authored-by: te-chan <t-nakanome@sakura-is.co.jp>
Co-authored-by: HQidea <HQidea@users.noreply.github.com>
Co-authored-by: Joshbly <36315710+Joshbly@users.noreply.github.com>
Co-authored-by: xhe <xw897002528@gmail.com>
Co-authored-by: weiwenyan-dev <154779315+weiwenyan-dev@users.noreply.github.com>
Co-authored-by: ex_wenyan.wei <ex_wenyan.wei@tcl.com>
Co-authored-by: engchina <12236799+engchina@users.noreply.github.com>
Co-authored-by: engchina <atjapan2015@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: 呆萌闷油瓶 <253605712@qq.com>
Co-authored-by: Kemal <kemalmeler@outlook.com>
Co-authored-by: Lazy_Frog <4590648+lazyFrogLOL@users.noreply.github.com>
Co-authored-by: Yi Xiao <54782454+YIXIAO0@users.noreply.github.com>
Co-authored-by: Steven sun <98230804+Tuyohai@users.noreply.github.com>
Co-authored-by: steven <sunzwj@digitalchina.com>
Co-authored-by: Kalo Chin <91766386+fdb02983rhy@users.noreply.github.com>
Co-authored-by: Katy Tao <34019945+KatyTao@users.noreply.github.com>
Co-authored-by: depy <42985524+h4ckdepy@users.noreply.github.com>
Co-authored-by: 胡春东 <gycm520@gmail.com>
Co-authored-by: Junjie.M <118170653@qq.com>
Co-authored-by: MuYu <mr.muzea@gmail.com>
Co-authored-by: Naoki Takashima <39912547+takatea@users.noreply.github.com>
Co-authored-by: Summer-Gu <37869445+gubinjie@users.noreply.github.com>
Co-authored-by: Fei He <droxer.he@gmail.com>
Co-authored-by: ybalbert001 <120714773+ybalbert001@users.noreply.github.com>
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
Co-authored-by: douxc <7553076+douxc@users.noreply.github.com>
Co-authored-by: liuzhenghua <1090179900@qq.com>
Co-authored-by: Wu Jiayang <62842862+Wu-Jiayang@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: kimjion <45935338+kimjion@users.noreply.github.com>
Co-authored-by: AugNSo <song.tiankai@icloud.com>
Co-authored-by: llinvokerl <38915183+llinvokerl@users.noreply.github.com>
Co-authored-by: liusurong.lsr <liusurong.lsr@alibaba-inc.com>
Co-authored-by: Vasu Negi <vasu-negi@users.noreply.github.com>
Co-authored-by: Hundredwz <1808096180@qq.com>
Co-authored-by: Xiyuan Chen <52963600+GareArc@users.noreply.github.com>

api/core/plugin/backwards_invocation/app.py

@@ -0,0 +1,189 @@
+from collections.abc import Generator, Mapping
+from typing import Optional, Union
+
+from controllers.service_api.wraps import create_or_update_end_user_for_user_id
+from core.app.apps.advanced_chat.app_generator import AdvancedChatAppGenerator
+from core.app.apps.agent_chat.app_generator import AgentChatAppGenerator
+from core.app.apps.chat.app_generator import ChatAppGenerator
+from core.app.apps.completion.app_generator import CompletionAppGenerator
+from core.app.apps.workflow.app_generator import WorkflowAppGenerator
+from core.app.entities.app_invoke_entities import InvokeFrom
+from core.plugin.backwards_invocation.base import BaseBackwardsInvocation
+from extensions.ext_database import db
+from models.account import Account
+from models.model import App, AppMode, EndUser
+
+
+class PluginAppBackwardsInvocation(BaseBackwardsInvocation):
+    @classmethod
+    def invoke_app(
+        cls,
+        app_id: str,
+        user_id: str,
+        tenant_id: str,
+        conversation_id: Optional[str],
+        query: Optional[str],
+        stream: bool,
+        inputs: Mapping,
+        files: list[dict],
+    ) -> Generator[Mapping | str, None, None] | Mapping:
+        """
+        invoke app
+        """
+        app = cls._get_app(app_id, tenant_id)
+        if not user_id:
+            user = create_or_update_end_user_for_user_id(app)
+        else:
+            user = cls._get_user(user_id)
+
+        conversation_id = conversation_id or ""
+
+        if app.mode in {AppMode.ADVANCED_CHAT.value, AppMode.AGENT_CHAT.value, AppMode.CHAT.value}:
+            if not query:
+                raise ValueError("missing query")
+
+            return cls.invoke_chat_app(app, user, conversation_id, query, stream, inputs, files)
+        elif app.mode == AppMode.WORKFLOW.value:
+            return cls.invoke_workflow_app(app, user, stream, inputs, files)
+        elif app.mode == AppMode.COMPLETION:
+            return cls.invoke_completion_app(app, user, stream, inputs, files)
+
+        raise ValueError("unexpected app type")
+
+    @classmethod
+    def invoke_chat_app(
+        cls,
+        app: App,
+        user: Account | EndUser,
+        conversation_id: str,
+        query: str,
+        stream: bool,
+        inputs: Mapping,
+        files: list[dict],
+    ) -> Generator[Mapping | str, None, None] | Mapping:
+        """
+        invoke chat app
+        """
+        if app.mode == AppMode.ADVANCED_CHAT.value:
+            workflow = app.workflow
+            if not workflow:
+                raise ValueError("unexpected app type")
+
+            return AdvancedChatAppGenerator().generate(
+                app_model=app,
+                workflow=workflow,
+                user=user,
+                args={
+                    "inputs": inputs,
+                    "query": query,
+                    "files": files,
+                    "conversation_id": conversation_id,
+                },
+                invoke_from=InvokeFrom.SERVICE_API,
+                streaming=stream,
+            )
+        elif app.mode == AppMode.AGENT_CHAT.value:
+            return AgentChatAppGenerator().generate(
+                app_model=app,
+                user=user,
+                args={
+                    "inputs": inputs,
+                    "query": query,
+                    "files": files,
+                    "conversation_id": conversation_id,
+                },
+                invoke_from=InvokeFrom.SERVICE_API,
+                streaming=stream,
+            )
+        elif app.mode == AppMode.CHAT.value:
+            return ChatAppGenerator().generate(
+                app_model=app,
+                user=user,
+                args={
+                    "inputs": inputs,
+                    "query": query,
+                    "files": files,
+                    "conversation_id": conversation_id,
+                },
+                invoke_from=InvokeFrom.SERVICE_API,
+                streaming=stream,
+            )
+        else:
+            raise ValueError("unexpected app type")
+
+    @classmethod
+    def invoke_workflow_app(
+        cls,
+        app: App,
+        user: EndUser | Account,
+        stream: bool,
+        inputs: Mapping,
+        files: list[dict],
+    ) -> Generator[Mapping | str, None, None] | Mapping:
+        """
+        invoke workflow app
+        """
+        workflow = app.workflow
+        if not workflow:
+            raise ValueError("")
+
+        return WorkflowAppGenerator().generate(
+            app_model=app,
+            workflow=workflow,
+            user=user,
+            args={"inputs": inputs, "files": files},
+            invoke_from=InvokeFrom.SERVICE_API,
+            streaming=stream,
+            call_depth=1,
+            workflow_thread_pool_id=None,
+        )
+
+    @classmethod
+    def invoke_completion_app(
+        cls,
+        app: App,
+        user: EndUser | Account,
+        stream: bool,
+        inputs: Mapping,
+        files: list[dict],
+    ) -> Generator[Mapping | str, None, None] | Mapping:
+        """
+        invoke completion app
+        """
+        return CompletionAppGenerator().generate(
+            app_model=app,
+            user=user,
+            args={"inputs": inputs, "files": files},
+            invoke_from=InvokeFrom.SERVICE_API,
+            streaming=stream,
+        )
+
+    @classmethod
+    def _get_user(cls, user_id: str) -> Union[EndUser, Account]:
+        """
+        get the user by user id
+        """
+
+        user = db.session.query(EndUser).filter(EndUser.id == user_id).first()
+        if not user:
+            user = db.session.query(Account).filter(Account.id == user_id).first()
+
+        if not user:
+            raise ValueError("user not found")
+
+        return user
+
+    @classmethod
+    def _get_app(cls, app_id: str, tenant_id: str) -> App:
+        """
+        get app
+        """
+        try:
+            app = db.session.query(App).filter(App.id == app_id).filter(App.tenant_id == tenant_id).first()
+        except Exception:
+            raise ValueError("app not found")
+
+        if not app:
+            raise ValueError("app not found")
+
+        return app

api/core/plugin/backwards_invocation/base.py

@@ -0,0 +1,29 @@
+from collections.abc import Generator, Mapping
+from typing import Generic, Optional, TypeVar
+
+from pydantic import BaseModel
+
+
+class BaseBackwardsInvocation:
+    @classmethod
+    def convert_to_event_stream(cls, response: Generator[BaseModel | Mapping | str, None, None] | BaseModel | Mapping):
+        if isinstance(response, Generator):
+            try:
+                for chunk in response:
+                    if isinstance(chunk, BaseModel | dict):
+                        yield BaseBackwardsInvocationResponse(data=chunk).model_dump_json().encode() + b"\n\n"
+                    elif isinstance(chunk, str):
+                        yield f"event: {chunk}\n\n".encode()
+            except Exception as e:
+                error_message = BaseBackwardsInvocationResponse(error=str(e)).model_dump_json()
+                yield f"{error_message}\n\n".encode()
+        else:
+            yield BaseBackwardsInvocationResponse(data=response).model_dump_json().encode() + b"\n\n"
+
+
+T = TypeVar("T", bound=dict | Mapping | str | bool | int | BaseModel)
+
+
+class BaseBackwardsInvocationResponse(BaseModel, Generic[T]):
+    data: Optional[T] = None
+    error: str = ""

api/core/plugin/backwards_invocation/encrypt.py

@@ -0,0 +1,30 @@
+from core.plugin.entities.request import RequestInvokeEncrypt
+from core.tools.utils.configuration import ProviderConfigEncrypter
+from models.account import Tenant
+
+
+class PluginEncrypter:
+    @classmethod
+    def invoke_encrypt(cls, tenant: Tenant, payload: RequestInvokeEncrypt) -> dict:
+        encrypter = ProviderConfigEncrypter(
+            tenant_id=tenant.id,
+            config=payload.config,
+            provider_type=payload.namespace,
+            provider_identity=payload.identity,
+        )
+
+        if payload.opt == "encrypt":
+            return {
+                "data": encrypter.encrypt(payload.data),
+            }
+        elif payload.opt == "decrypt":
+            return {
+                "data": encrypter.decrypt(payload.data),
+            }
+        elif payload.opt == "clear":
+            encrypter.delete_tool_credentials_cache()
+            return {
+                "data": {},
+            }
+        else:
+            raise ValueError(f"Invalid opt: {payload.opt}")

api/core/plugin/backwards_invocation/model.py

@@ -0,0 +1,321 @@
+import tempfile
+from binascii import hexlify, unhexlify
+from collections.abc import Generator
+
+from core.model_manager import ModelManager
+from core.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk
+from core.model_runtime.entities.message_entities import (
+    PromptMessage,
+    SystemPromptMessage,
+    UserPromptMessage,
+)
+from core.plugin.backwards_invocation.base import BaseBackwardsInvocation
+from core.plugin.entities.request import (
+    RequestInvokeLLM,
+    RequestInvokeModeration,
+    RequestInvokeRerank,
+    RequestInvokeSpeech2Text,
+    RequestInvokeSummary,
+    RequestInvokeTextEmbedding,
+    RequestInvokeTTS,
+)
+from core.tools.entities.tool_entities import ToolProviderType
+from core.tools.utils.model_invocation_utils import ModelInvocationUtils
+from core.workflow.nodes.llm.node import LLMNode
+from models.account import Tenant
+
+
+class PluginModelBackwardsInvocation(BaseBackwardsInvocation):
+    @classmethod
+    def invoke_llm(
+        cls, user_id: str, tenant: Tenant, payload: RequestInvokeLLM
+    ) -> Generator[LLMResultChunk, None, None] | LLMResult:
+        """
+        invoke llm
+        """
+        model_instance = ModelManager().get_model_instance(
+            tenant_id=tenant.id,
+            provider=payload.provider,
+            model_type=payload.model_type,
+            model=payload.model,
+        )
+
+        # invoke model
+        response = model_instance.invoke_llm(
+            prompt_messages=payload.prompt_messages,
+            model_parameters=payload.completion_params,
+            tools=payload.tools,
+            stop=payload.stop,
+            stream=payload.stream or True,
+            user=user_id,
+        )
+
+        if isinstance(response, Generator):
+
+            def handle() -> Generator[LLMResultChunk, None, None]:
+                for chunk in response:
+                    if chunk.delta.usage:
+                        LLMNode.deduct_llm_quota(
+                            tenant_id=tenant.id, model_instance=model_instance, usage=chunk.delta.usage
+                        )
+                    yield chunk
+
+            return handle()
+        else:
+            if response.usage:
+                LLMNode.deduct_llm_quota(tenant_id=tenant.id, model_instance=model_instance, usage=response.usage)
+            return response
+
+    @classmethod
+    def invoke_text_embedding(cls, user_id: str, tenant: Tenant, payload: RequestInvokeTextEmbedding):
+        """
+        invoke text embedding
+        """
+        model_instance = ModelManager().get_model_instance(
+            tenant_id=tenant.id,
+            provider=payload.provider,
+            model_type=payload.model_type,
+            model=payload.model,
+        )
+
+        # invoke model
+        response = model_instance.invoke_text_embedding(
+            texts=payload.texts,
+            user=user_id,
+        )
+
+        return response
+
+    @classmethod
+    def invoke_rerank(cls, user_id: str, tenant: Tenant, payload: RequestInvokeRerank):
+        """
+        invoke rerank
+        """
+        model_instance = ModelManager().get_model_instance(
+            tenant_id=tenant.id,
+            provider=payload.provider,
+            model_type=payload.model_type,
+            model=payload.model,
+        )
+
+        # invoke model
+        response = model_instance.invoke_rerank(
+            query=payload.query,
+            docs=payload.docs,
+            score_threshold=payload.score_threshold,
+            top_n=payload.top_n,
+            user=user_id,
+        )
+
+        return response
+
+    @classmethod
+    def invoke_tts(cls, user_id: str, tenant: Tenant, payload: RequestInvokeTTS):
+        """
+        invoke tts
+        """
+        model_instance = ModelManager().get_model_instance(
+            tenant_id=tenant.id,
+            provider=payload.provider,
+            model_type=payload.model_type,
+            model=payload.model,
+        )
+
+        # invoke model
+        response = model_instance.invoke_tts(
+            content_text=payload.content_text,
+            tenant_id=tenant.id,
+            voice=payload.voice,
+            user=user_id,
+        )
+
+        def handle() -> Generator[dict, None, None]:
+            for chunk in response:
+                yield {"result": hexlify(chunk).decode("utf-8")}
+
+        return handle()
+
+    @classmethod
+    def invoke_speech2text(cls, user_id: str, tenant: Tenant, payload: RequestInvokeSpeech2Text):
+        """
+        invoke speech2text
+        """
+        model_instance = ModelManager().get_model_instance(
+            tenant_id=tenant.id,
+            provider=payload.provider,
+            model_type=payload.model_type,
+            model=payload.model,
+        )
+
+        # invoke model
+        with tempfile.NamedTemporaryFile(suffix=".mp3", mode="wb", delete=True) as temp:
+            temp.write(unhexlify(payload.file))
+            temp.flush()
+            temp.seek(0)
+
+            response = model_instance.invoke_speech2text(
+                file=temp,
+                user=user_id,
+            )
+
+            return {
+                "result": response,
+            }
+
+    @classmethod
+    def invoke_moderation(cls, user_id: str, tenant: Tenant, payload: RequestInvokeModeration):
+        """
+        invoke moderation
+        """
+        model_instance = ModelManager().get_model_instance(
+            tenant_id=tenant.id,
+            provider=payload.provider,
+            model_type=payload.model_type,
+            model=payload.model,
+        )
+
+        # invoke model
+        response = model_instance.invoke_moderation(
+            text=payload.text,
+            user=user_id,
+        )
+
+        return {
+            "result": response,
+        }
+
+    @classmethod
+    def get_system_model_max_tokens(cls, tenant_id: str) -> int:
+        """
+        get system model max tokens
+        """
+        return ModelInvocationUtils.get_max_llm_context_tokens(tenant_id=tenant_id)
+
+    @classmethod
+    def get_prompt_tokens(cls, tenant_id: str, prompt_messages: list[PromptMessage]) -> int:
+        """
+        get prompt tokens
+        """
+        return ModelInvocationUtils.calculate_tokens(tenant_id=tenant_id, prompt_messages=prompt_messages)
+
+    @classmethod
+    def invoke_system_model(
+        cls,
+        user_id: str,
+        tenant: Tenant,
+        prompt_messages: list[PromptMessage],
+    ) -> LLMResult:
+        """
+        invoke system model
+        """
+        return ModelInvocationUtils.invoke(
+            user_id=user_id,
+            tenant_id=tenant.id,
+            tool_type=ToolProviderType.PLUGIN,
+            tool_name="plugin",
+            prompt_messages=prompt_messages,
+        )
+
+    @classmethod
+    def invoke_summary(cls, user_id: str, tenant: Tenant, payload: RequestInvokeSummary):
+        """
+        invoke summary
+        """
+        max_tokens = cls.get_system_model_max_tokens(tenant_id=tenant.id)
+        content = payload.text
+
+        SUMMARY_PROMPT = """You are a professional language researcher, you are interested in the language
+and you can quickly aimed at the main point of an webpage and reproduce it in your own words but 
+retain the original meaning and keep the key points. 
+however, the text you got is too long, what you got is possible a part of the text.
+Please summarize the text you got.
+
+Here is the extra instruction you need to follow:
+<extra_instruction>
+{payload.instruction}
+</extra_instruction>
+"""
+
+        if (
+            cls.get_prompt_tokens(
+                tenant_id=tenant.id,
+                prompt_messages=[UserPromptMessage(content=content)],
+            )
+            < max_tokens * 0.6
+        ):
+            return content
+
+        def get_prompt_tokens(content: str) -> int:
+            return cls.get_prompt_tokens(
+                tenant_id=tenant.id,
+                prompt_messages=[
+                    SystemPromptMessage(content=SUMMARY_PROMPT.replace("{payload.instruction}", payload.instruction)),
+                    UserPromptMessage(content=content),
+                ],
+            )
+
+        def summarize(content: str) -> str:
+            summary = cls.invoke_system_model(
+                user_id=user_id,
+                tenant=tenant,
+                prompt_messages=[
+                    SystemPromptMessage(content=SUMMARY_PROMPT.replace("{payload.instruction}", payload.instruction)),
+                    UserPromptMessage(content=content),
+                ],
+            )
+
+            assert isinstance(summary.message.content, str)
+            return summary.message.content
+
+        lines = content.split("\n")
+        new_lines: list[str] = []
+        # split long line into multiple lines
+        for i in range(len(lines)):
+            line = lines[i]
+            if not line.strip():
+                continue
+            if len(line) < max_tokens * 0.5:
+                new_lines.append(line)
+            elif get_prompt_tokens(line) > max_tokens * 0.7:
+                while get_prompt_tokens(line) > max_tokens * 0.7:
+                    new_lines.append(line[: int(max_tokens * 0.5)])
+                    line = line[int(max_tokens * 0.5) :]
+                new_lines.append(line)
+            else:
+                new_lines.append(line)
+
+        # merge lines into messages with max tokens
+        messages: list[str] = []
+        for i in new_lines:  # type: ignore
+            if len(messages) == 0:
+                messages.append(i)  # type: ignore
+            else:
+                if len(messages[-1]) + len(i) < max_tokens * 0.5:  # type: ignore
+                    messages[-1] += i  # type: ignore
+                if get_prompt_tokens(messages[-1] + i) > max_tokens * 0.7:  # type: ignore
+                    messages.append(i)  # type: ignore
+                else:
+                    messages[-1] += i  # type: ignore
+
+        summaries = []
+        for i in range(len(messages)):
+            message = messages[i]
+            summary = summarize(message)
+            summaries.append(summary)
+
+        result = "\n".join(summaries)
+
+        if (
+            cls.get_prompt_tokens(
+                tenant_id=tenant.id,
+                prompt_messages=[UserPromptMessage(content=result)],
+            )
+            > max_tokens * 0.7
+        ):
+            return cls.invoke_summary(
+                user_id=user_id,
+                tenant=tenant,
+                payload=RequestInvokeSummary(text=result, instruction=payload.instruction),
+            )
+
+        return result

api/core/plugin/backwards_invocation/node.py

@@ -0,0 +1,117 @@
+from core.plugin.backwards_invocation.base import BaseBackwardsInvocation
+from core.workflow.nodes.enums import NodeType
+from core.workflow.nodes.parameter_extractor.entities import (
+    ModelConfig as ParameterExtractorModelConfig,
+)
+from core.workflow.nodes.parameter_extractor.entities import (
+    ParameterConfig,
+    ParameterExtractorNodeData,
+)
+from core.workflow.nodes.question_classifier.entities import (
+    ClassConfig,
+    QuestionClassifierNodeData,
+)
+from core.workflow.nodes.question_classifier.entities import (
+    ModelConfig as QuestionClassifierModelConfig,
+)
+from services.workflow_service import WorkflowService
+
+
+class PluginNodeBackwardsInvocation(BaseBackwardsInvocation):
+    @classmethod
+    def invoke_parameter_extractor(
+        cls,
+        tenant_id: str,
+        user_id: str,
+        parameters: list[ParameterConfig],
+        model_config: ParameterExtractorModelConfig,
+        instruction: str,
+        query: str,
+    ) -> dict:
+        """
+        Invoke parameter extractor node.
+
+        :param tenant_id: str
+        :param user_id: str
+        :param parameters: list[ParameterConfig]
+        :param model_config: ModelConfig
+        :param instruction: str
+        :param query: str
+        :return: dict
+        """
+        workflow_service = WorkflowService()
+        node_id = "1919810"
+        node_data = ParameterExtractorNodeData(
+            title="parameter_extractor",
+            desc="parameter_extractor",
+            parameters=parameters,
+            reasoning_mode="function_call",
+            query=[node_id, "query"],
+            model=model_config,
+            instruction=instruction,  # instruct with variables are not supported
+        )
+        node_data_dict = node_data.model_dump()
+        node_data_dict["type"] = NodeType.PARAMETER_EXTRACTOR.value
+        execution = workflow_service.run_free_workflow_node(
+            node_data_dict,
+            tenant_id=tenant_id,
+            user_id=user_id,
+            node_id=node_id,
+            user_inputs={
+                f"{node_id}.query": query,
+            },
+        )
+
+        return {
+            "inputs": execution.inputs_dict,
+            "outputs": execution.outputs_dict,
+            "process_data": execution.process_data_dict,
+        }
+
+    @classmethod
+    def invoke_question_classifier(
+        cls,
+        tenant_id: str,
+        user_id: str,
+        model_config: QuestionClassifierModelConfig,
+        classes: list[ClassConfig],
+        instruction: str,
+        query: str,
+    ) -> dict:
+        """
+        Invoke question classifier node.
+
+        :param tenant_id: str
+        :param user_id: str
+        :param model_config: ModelConfig
+        :param classes: list[ClassConfig]
+        :param instruction: str
+        :param query: str
+        :return: dict
+        """
+        workflow_service = WorkflowService()
+        node_id = "1919810"
+        node_data = QuestionClassifierNodeData(
+            title="question_classifier",
+            desc="question_classifier",
+            query_variable_selector=[node_id, "query"],
+            model=model_config,
+            classes=classes,
+            instruction=instruction,  # instruct with variables are not supported
+        )
+        node_data_dict = node_data.model_dump()
+        execution = workflow_service.run_free_workflow_node(
+            node_data_dict,
+            tenant_id=tenant_id,
+            user_id=user_id,
+            node_id=node_id,
+            user_inputs={
+                f"{node_id}.query": query,
+            },
+        )
+
+        return {
+            "inputs": execution.inputs_dict,
+            "outputs": execution.outputs_dict,
+            "process_data": execution.process_data_dict,
+        }

api/core/plugin/backwards_invocation/tool.py

@@ -0,0 +1,45 @@
+from collections.abc import Generator
+from typing import Any
+
+from core.callback_handler.workflow_tool_callback_handler import DifyWorkflowCallbackHandler
+from core.plugin.backwards_invocation.base import BaseBackwardsInvocation
+from core.tools.entities.tool_entities import ToolInvokeMessage, ToolProviderType
+from core.tools.tool_engine import ToolEngine
+from core.tools.tool_manager import ToolManager
+from core.tools.utils.message_transformer import ToolFileMessageTransformer
+
+
+class PluginToolBackwardsInvocation(BaseBackwardsInvocation):
+    """
+    Backwards invocation for plugin tools.
+    """
+
+    @classmethod
+    def invoke_tool(
+        cls,
+        tenant_id: str,
+        user_id: str,
+        tool_type: ToolProviderType,
+        provider: str,
+        tool_name: str,
+        tool_parameters: dict[str, Any],
+    ) -> Generator[ToolInvokeMessage, None, None]:
+        """
+        invoke tool
+        """
+        # get tool runtime
+        try:
+            tool_runtime = ToolManager.get_tool_runtime_from_plugin(
+                tool_type, tenant_id, provider, tool_name, tool_parameters
+            )
+            response = ToolEngine.generic_invoke(
+                tool_runtime, tool_parameters, user_id, DifyWorkflowCallbackHandler(), workflow_call_depth=1
+            )
+
+            response = ToolFileMessageTransformer.transform_tool_invoke_messages(
+                response, user_id=user_id, tenant_id=tenant_id
+            )
+
+            return response
+        except Exception as e:
+            raise e

api/core/plugin/entities/base.py

@@ -0,0 +1,9 @@
+from datetime import datetime
+
+from pydantic import BaseModel
+
+
+class BasePluginEntity(BaseModel):
+    id: str
+    created_at: datetime
+    updated_at: datetime

api/core/plugin/entities/bundle.py

@@ -0,0 +1,30 @@
+from enum import StrEnum
+
+from pydantic import BaseModel
+
+from core.plugin.entities.plugin import PluginDeclaration, PluginInstallationSource
+
+
+class PluginBundleDependency(BaseModel):
+    class Type(StrEnum):
+        Github = PluginInstallationSource.Github.value
+        Marketplace = PluginInstallationSource.Marketplace.value
+        Package = PluginInstallationSource.Package.value
+
+    class Github(BaseModel):
+        repo_address: str
+        repo: str
+        release: str
+        packages: str
+
+    class Marketplace(BaseModel):
+        organization: str
+        plugin: str
+        version: str
+
+    class Package(BaseModel):
+        unique_identifier: str
+        manifest: PluginDeclaration
+
+    type: Type
+    value: Github | Marketplace | Package

api/core/plugin/entities/endpoint.py

@@ -0,0 +1,54 @@
+from datetime import datetime
+from typing import Optional
+
+from pydantic import BaseModel, Field, model_validator
+
+from configs import dify_config
+from core.entities.provider_entities import ProviderConfig
+from core.plugin.entities.base import BasePluginEntity
+
+
+class EndpointDeclaration(BaseModel):
+    """
+    declaration of an endpoint
+    """
+
+    path: str
+    method: str
+    hidden: bool = Field(default=False)
+
+
+class EndpointProviderDeclaration(BaseModel):
+    """
+    declaration of an endpoint group
+    """
+
+    settings: list[ProviderConfig] = Field(default_factory=list)
+    endpoints: Optional[list[EndpointDeclaration]] = Field(default_factory=list)
+
+
+class EndpointEntity(BasePluginEntity):
+    """
+    entity of an endpoint
+    """
+
+    settings: dict
+    tenant_id: str
+    plugin_id: str
+    expired_at: datetime
+    declaration: EndpointProviderDeclaration = Field(default_factory=EndpointProviderDeclaration)
+
+
+class EndpointEntityWithInstance(EndpointEntity):
+    name: str
+    enabled: bool
+    url: str
+    hook_id: str
+
+    @model_validator(mode="before")
+    @classmethod
+    def render_url_template(cls, values):
+        if "url" not in values:
+            url_template = dify_config.ENDPOINT_URL_TEMPLATE
+            values["url"] = url_template.replace("{hook_id}", values["hook_id"])
+        return values

api/core/plugin/entities/marketplace.py

@@ -0,0 +1,45 @@
+from typing import Optional
+
+from pydantic import BaseModel, Field, model_validator
+
+from core.model_runtime.entities.provider_entities import ProviderEntity
+from core.plugin.entities.endpoint import EndpointProviderDeclaration
+from core.plugin.entities.plugin import PluginResourceRequirements
+from core.tools.entities.common_entities import I18nObject
+from core.tools.entities.tool_entities import ToolProviderEntity
+
+
+class MarketplacePluginDeclaration(BaseModel):
+    name: str = Field(..., description="Unique identifier for the plugin within the marketplace")
+    org: str = Field(..., description="Organization or developer responsible for creating and maintaining the plugin")
+    plugin_id: str = Field(..., description="Globally unique identifier for the plugin across all marketplaces")
+    icon: str = Field(..., description="URL or path to the plugin's visual representation")
+    label: I18nObject = Field(..., description="Localized display name for the plugin in different languages")
+    brief: I18nObject = Field(..., description="Short, localized description of the plugin's functionality")
+    resource: PluginResourceRequirements = Field(
+        ..., description="Specification of computational resources needed to run the plugin"
+    )
+    endpoint: Optional[EndpointProviderDeclaration] = Field(
+        None, description="Configuration for the plugin's API endpoint, if applicable"
+    )
+    model: Optional[ProviderEntity] = Field(None, description="Details of the AI model used by the plugin, if any")
+    tool: Optional[ToolProviderEntity] = Field(
+        None, description="Information about the tool functionality provided by the plugin, if any"
+    )
+    latest_version: str = Field(
+        ..., description="Most recent version number of the plugin available in the marketplace"
+    )
+    latest_package_identifier: str = Field(
+        ..., description="Unique identifier for the latest package release of the plugin"
+    )
+
+    @model_validator(mode="before")
+    @classmethod
+    def transform_declaration(cls, data: dict):
+        if "endpoint" in data and not data["endpoint"]:
+            del data["endpoint"]
+        if "model" in data and not data["model"]:
+            del data["model"]
+        if "tool" in data and not data["tool"]:
+            del data["tool"]
+        return data

api/core/plugin/entities/parameters.py

@@ -0,0 +1,162 @@
+import enum
+from typing import Any, Optional, Union
+
+from pydantic import BaseModel, Field, field_validator
+
+from core.entities.parameter_entities import CommonParameterType
+from core.tools.entities.common_entities import I18nObject
+
+
+class PluginParameterOption(BaseModel):
+    value: str = Field(..., description="The value of the option")
+    label: I18nObject = Field(..., description="The label of the option")
+
+    @field_validator("value", mode="before")
+    @classmethod
+    def transform_id_to_str(cls, value) -> str:
+        if not isinstance(value, str):
+            return str(value)
+        else:
+            return value
+
+
+class PluginParameterType(enum.StrEnum):
+    """
+    all available parameter types
+    """
+
+    STRING = CommonParameterType.STRING.value
+    NUMBER = CommonParameterType.NUMBER.value
+    BOOLEAN = CommonParameterType.BOOLEAN.value
+    SELECT = CommonParameterType.SELECT.value
+    SECRET_INPUT = CommonParameterType.SECRET_INPUT.value
+    FILE = CommonParameterType.FILE.value
+    FILES = CommonParameterType.FILES.value
+    APP_SELECTOR = CommonParameterType.APP_SELECTOR.value
+    MODEL_SELECTOR = CommonParameterType.MODEL_SELECTOR.value
+    TOOLS_SELECTOR = CommonParameterType.TOOLS_SELECTOR.value
+
+    # deprecated, should not use.
+    SYSTEM_FILES = CommonParameterType.SYSTEM_FILES.value
+
+
+class PluginParameterAutoGenerate(BaseModel):
+    class Type(enum.StrEnum):
+        PROMPT_INSTRUCTION = "prompt_instruction"
+
+    type: Type
+
+
+class PluginParameterTemplate(BaseModel):
+    enabled: bool = Field(default=False, description="Whether the parameter is jinja enabled")
+
+
+class PluginParameter(BaseModel):
+    name: str = Field(..., description="The name of the parameter")
+    label: I18nObject = Field(..., description="The label presented to the user")
+    placeholder: Optional[I18nObject] = Field(default=None, description="The placeholder presented to the user")
+    scope: str | None = None
+    auto_generate: Optional[PluginParameterAutoGenerate] = None
+    template: Optional[PluginParameterTemplate] = None
+    required: bool = False
+    default: Optional[Union[float, int, str]] = None
+    min: Optional[Union[float, int]] = None
+    max: Optional[Union[float, int]] = None
+    precision: Optional[int] = None
+    options: list[PluginParameterOption] = Field(default_factory=list)
+
+    @field_validator("options", mode="before")
+    @classmethod
+    def transform_options(cls, v):
+        if not isinstance(v, list):
+            return []
+        return v
+
+
+def as_normal_type(typ: enum.StrEnum):
+    if typ.value in {
+        PluginParameterType.SECRET_INPUT,
+        PluginParameterType.SELECT,
+    }:
+        return "string"
+    return typ.value
+
+
+def cast_parameter_value(typ: enum.StrEnum, value: Any, /):
+    try:
+        match typ.value:
+            case PluginParameterType.STRING | PluginParameterType.SECRET_INPUT | PluginParameterType.SELECT:
+                if value is None:
+                    return ""
+                else:
+                    return value if isinstance(value, str) else str(value)
+
+            case PluginParameterType.BOOLEAN:
+                if value is None:
+                    return False
+                elif isinstance(value, str):
+                    # Allowed YAML boolean value strings: https://yaml.org/type/bool.html
+                    # and also '0' for False and '1' for True
+                    match value.lower():
+                        case "true" | "yes" | "y" | "1":
+                            return True
+                        case "false" | "no" | "n" | "0":
+                            return False
+                        case _:
+                            return bool(value)
+                else:
+                    return value if isinstance(value, bool) else bool(value)
+
+            case PluginParameterType.NUMBER:
+                if isinstance(value, int | float):
+                    return value
+                elif isinstance(value, str) and value:
+                    if "." in value:
+                        return float(value)
+                    else:
+                        return int(value)
+            case PluginParameterType.SYSTEM_FILES | PluginParameterType.FILES:
+                if not isinstance(value, list):
+                    return [value]
+                return value
+            case PluginParameterType.FILE:
+                if isinstance(value, list):
+                    if len(value) != 1:
+                        raise ValueError("This parameter only accepts one file but got multiple files while invoking.")
+                    else:
+                        return value[0]
+                return value
+            case PluginParameterType.MODEL_SELECTOR | PluginParameterType.APP_SELECTOR:
+                if not isinstance(value, dict):
+                    raise ValueError("The selector must be a dictionary.")
+                return value
+            case PluginParameterType.TOOLS_SELECTOR:
+                if not isinstance(value, list):
+                    raise ValueError("The tools selector must be a list.")
+                return value
+            case _:
+                return str(value)
+    except ValueError:
+        raise
+    except Exception:
+        raise ValueError(f"The tool parameter value {value} is not in correct type of {as_normal_type(typ)}.")
+
+
+def init_frontend_parameter(rule: PluginParameter, type: enum.StrEnum, value: Any):
+    """
+    init frontend parameter by rule
+    """
+    parameter_value = value
+    if not parameter_value and parameter_value != 0:
+        # get default value
+        parameter_value = rule.default
+        if not parameter_value and rule.required:
+            raise ValueError(f"tool parameter {rule.name} not found in tool config")
+
+    if type == PluginParameterType.SELECT:
+        # check if tool_parameter_config in options
+        options = [x.value for x in rule.options]
+        if parameter_value is not None and parameter_value not in options:
+            raise ValueError(f"tool parameter {rule.name} value {parameter_value} not in options {options}")
+
+    return cast_parameter_value(type, parameter_value)

api/core/plugin/entities/plugin.py

@@ -0,0 +1,220 @@
+import datetime
+import enum
+import re
+from collections.abc import Mapping
+from typing import Any, Optional
+
+from pydantic import BaseModel, Field, model_validator
+
+from core.agent.plugin_entities import AgentStrategyProviderEntity
+from core.model_runtime.entities.provider_entities import ProviderEntity
+from core.plugin.entities.base import BasePluginEntity
+from core.plugin.entities.endpoint import EndpointProviderDeclaration
+from core.tools.entities.common_entities import I18nObject
+from core.tools.entities.tool_entities import ToolProviderEntity
+
+
+class PluginInstallationSource(enum.StrEnum):
+    Github = "github"
+    Marketplace = "marketplace"
+    Package = "package"
+    Remote = "remote"
+
+
+class PluginResourceRequirements(BaseModel):
+    memory: int
+
+    class Permission(BaseModel):
+        class Tool(BaseModel):
+            enabled: Optional[bool] = Field(default=False)
+
+        class Model(BaseModel):
+            enabled: Optional[bool] = Field(default=False)
+            llm: Optional[bool] = Field(default=False)
+            text_embedding: Optional[bool] = Field(default=False)
+            rerank: Optional[bool] = Field(default=False)
+            tts: Optional[bool] = Field(default=False)
+            speech2text: Optional[bool] = Field(default=False)
+            moderation: Optional[bool] = Field(default=False)
+
+        class Node(BaseModel):
+            enabled: Optional[bool] = Field(default=False)
+
+        class Endpoint(BaseModel):
+            enabled: Optional[bool] = Field(default=False)
+
+        class Storage(BaseModel):
+            enabled: Optional[bool] = Field(default=False)
+            size: int = Field(ge=1024, le=1073741824, default=1048576)
+
+        tool: Optional[Tool] = Field(default=None)
+        model: Optional[Model] = Field(default=None)
+        node: Optional[Node] = Field(default=None)
+        endpoint: Optional[Endpoint] = Field(default=None)
+        storage: Storage = Field(default=None)
+
+    permission: Optional[Permission] = Field(default=None)
+
+
+class PluginCategory(enum.StrEnum):
+    Tool = "tool"
+    Model = "model"
+    Extension = "extension"
+    AgentStrategy = "agent-strategy"
+
+
+class PluginDeclaration(BaseModel):
+    class Plugins(BaseModel):
+        tools: Optional[list[str]] = Field(default_factory=list)
+        models: Optional[list[str]] = Field(default_factory=list)
+        endpoints: Optional[list[str]] = Field(default_factory=list)
+
+    version: str = Field(..., pattern=r"^\d{1,4}(\.\d{1,4}){1,3}(-\w{1,16})?$")
+    author: Optional[str] = Field(..., pattern=r"^[a-zA-Z0-9_-]{1,64}$")
+    name: str = Field(..., pattern=r"^[a-z0-9_-]{1,128}$")
+    description: I18nObject
+    icon: str
+    label: I18nObject
+    category: PluginCategory
+    created_at: datetime.datetime
+    resource: PluginResourceRequirements
+    plugins: Plugins
+    tags: list[str] = Field(default_factory=list)
+    verified: bool = Field(default=False)
+    tool: Optional[ToolProviderEntity] = None
+    model: Optional[ProviderEntity] = None
+    endpoint: Optional[EndpointProviderDeclaration] = None
+    agent_strategy: Optional[AgentStrategyProviderEntity] = None
+
+    @model_validator(mode="before")
+    @classmethod
+    def validate_category(cls, values: dict) -> dict:
+        # auto detect category
+        if values.get("tool"):
+            values["category"] = PluginCategory.Tool
+        elif values.get("model"):
+            values["category"] = PluginCategory.Model
+        elif values.get("agent_strategy"):
+            values["category"] = PluginCategory.AgentStrategy
+        else:
+            values["category"] = PluginCategory.Extension
+        return values
+
+
+class PluginInstallation(BasePluginEntity):
+    tenant_id: str
+    endpoints_setups: int
+    endpoints_active: int
+    runtime_type: str
+    source: PluginInstallationSource
+    meta: Mapping[str, Any]
+    plugin_id: str
+    plugin_unique_identifier: str
+    version: str
+    checksum: str
+    declaration: PluginDeclaration
+
+
+class PluginEntity(PluginInstallation):
+    name: str
+    installation_id: str
+    version: str
+    latest_version: Optional[str] = None
+    latest_unique_identifier: Optional[str] = None
+
+    @model_validator(mode="after")
+    def set_plugin_id(self):
+        if self.declaration.tool:
+            self.declaration.tool.plugin_id = self.plugin_id
+        return self
+
+
+class GithubPackage(BaseModel):
+    repo: str
+    version: str
+    package: str
+
+
+class GithubVersion(BaseModel):
+    repo: str
+    version: str
+
+
+class GenericProviderID:
+    organization: str
+    plugin_name: str
+    provider_name: str
+    is_hardcoded: bool
+
+    def to_string(self) -> str:
+        return str(self)
+
+    def __str__(self) -> str:
+        return f"{self.organization}/{self.plugin_name}/{self.provider_name}"
+
+    def __init__(self, value: str, is_hardcoded: bool = False) -> None:
+        # check if the value is a valid plugin id with format: $organization/$plugin_name/$provider_name
+        if not re.match(r"^[a-z0-9_-]+\/[a-z0-9_-]+\/[a-z0-9_-]+$", value):
+            # check if matches [a-z0-9_-]+, if yes, append with langgenius/$value/$value
+            if re.match(r"^[a-z0-9_-]+$", value):
+                value = f"langgenius/{value}/{value}"
+            else:
+                raise ValueError(f"Invalid plugin id {value}")
+
+        self.organization, self.plugin_name, self.provider_name = value.split("/")
+        self.is_hardcoded = is_hardcoded
+
+    @property
+    def plugin_id(self) -> str:
+        return f"{self.organization}/{self.plugin_name}"
+
+
+class ModelProviderID(GenericProviderID):
+    def __init__(self, value: str, is_hardcoded: bool = False) -> None:
+        super().__init__(value, is_hardcoded)
+        if self.organization == "langgenius" and self.provider_name == "google":
+            self.plugin_name = "gemini"
+
+
+class ToolProviderID(GenericProviderID):
+    def __init__(self, value: str, is_hardcoded: bool = False) -> None:
+        super().__init__(value, is_hardcoded)
+        if self.organization == "langgenius":
+            if self.provider_name in ["jina", "siliconflow"]:
+                self.plugin_name = f"{self.provider_name}_tool"
+
+
+class PluginDependency(BaseModel):
+    class Type(enum.StrEnum):
+        Github = PluginInstallationSource.Github.value
+        Marketplace = PluginInstallationSource.Marketplace.value
+        Package = PluginInstallationSource.Package.value
+
+    class Github(BaseModel):
+        repo: str
+        version: str
+        package: str
+        github_plugin_unique_identifier: str
+
+        @property
+        def plugin_unique_identifier(self) -> str:
+            return self.github_plugin_unique_identifier
+
+    class Marketplace(BaseModel):
+        marketplace_plugin_unique_identifier: str
+
+        @property
+        def plugin_unique_identifier(self) -> str:
+            return self.marketplace_plugin_unique_identifier
+
+    class Package(BaseModel):
+        plugin_unique_identifier: str
+
+    type: Type
+    value: Github | Marketplace | Package
+    current_identifier: Optional[str] = None
+
+
+class MissingPluginDependency(BaseModel):
+    plugin_unique_identifier: str
+    current_identifier: Optional[str] = None

api/core/plugin/entities/plugin_daemon.py

@@ -0,0 +1,160 @@
+from datetime import datetime
+from enum import StrEnum
+from typing import Generic, Optional, TypeVar
+
+from pydantic import BaseModel, ConfigDict, Field
+
+from core.agent.plugin_entities import AgentProviderEntityWithPlugin
+from core.model_runtime.entities.model_entities import AIModelEntity
+from core.model_runtime.entities.provider_entities import ProviderEntity
+from core.plugin.entities.base import BasePluginEntity
+from core.plugin.entities.plugin import PluginDeclaration
+from core.tools.entities.common_entities import I18nObject
+from core.tools.entities.tool_entities import ToolProviderEntityWithPlugin
+
+T = TypeVar("T", bound=(BaseModel | dict | list | bool | str))
+
+
+class PluginDaemonBasicResponse(BaseModel, Generic[T]):
+    """
+    Basic response from plugin daemon.
+    """
+
+    code: int
+    message: str
+    data: Optional[T]
+
+
+class InstallPluginMessage(BaseModel):
+    """
+    Message for installing a plugin.
+    """
+
+    class Event(StrEnum):
+        Info = "info"
+        Done = "done"
+        Error = "error"
+
+    event: Event
+    data: str
+
+
+class PluginToolProviderEntity(BaseModel):
+    provider: str
+    plugin_unique_identifier: str
+    plugin_id: str
+    declaration: ToolProviderEntityWithPlugin
+
+
+class PluginAgentProviderEntity(BaseModel):
+    provider: str
+    plugin_unique_identifier: str
+    plugin_id: str
+    declaration: AgentProviderEntityWithPlugin
+
+
+class PluginBasicBooleanResponse(BaseModel):
+    """
+    Basic boolean response from plugin daemon.
+    """
+
+    result: bool
+    credentials: dict | None = None
+
+
+class PluginModelSchemaEntity(BaseModel):
+    model_schema: AIModelEntity = Field(description="The model schema.")
+
+    # pydantic configs
+    model_config = ConfigDict(protected_namespaces=())
+
+
+class PluginModelProviderEntity(BaseModel):
+    id: str = Field(description="ID")
+    created_at: datetime = Field(description="The created at time of the model provider.")
+    updated_at: datetime = Field(description="The updated at time of the model provider.")
+    provider: str = Field(description="The provider of the model.")
+    tenant_id: str = Field(description="The tenant ID.")
+    plugin_unique_identifier: str = Field(description="The plugin unique identifier.")
+    plugin_id: str = Field(description="The plugin ID.")
+    declaration: ProviderEntity = Field(description="The declaration of the model provider.")
+
+
+class PluginTextEmbeddingNumTokensResponse(BaseModel):
+    """
+    Response for number of tokens.
+    """
+
+    num_tokens: list[int] = Field(description="The number of tokens.")
+
+
+class PluginLLMNumTokensResponse(BaseModel):
+    """
+    Response for number of tokens.
+    """
+
+    num_tokens: int = Field(description="The number of tokens.")
+
+
+class PluginStringResultResponse(BaseModel):
+    result: str = Field(description="The result of the string.")
+
+
+class PluginVoiceEntity(BaseModel):
+    name: str = Field(description="The name of the voice.")
+    value: str = Field(description="The value of the voice.")
+
+
+class PluginVoicesResponse(BaseModel):
+    voices: list[PluginVoiceEntity] = Field(description="The result of the voices.")
+
+
+class PluginDaemonError(BaseModel):
+    """
+    Error from plugin daemon.
+    """
+
+    error_type: str
+    message: str
+
+
+class PluginDaemonInnerError(Exception):
+    code: int
+    message: str
+
+    def __init__(self, code: int, message: str):
+        self.code = code
+        self.message = message
+
+
+class PluginInstallTaskStatus(StrEnum):
+    Pending = "pending"
+    Running = "running"
+    Success = "success"
+    Failed = "failed"
+
+
+class PluginInstallTaskPluginStatus(BaseModel):
+    plugin_unique_identifier: str = Field(description="The plugin unique identifier of the install task.")
+    plugin_id: str = Field(description="The plugin ID of the install task.")
+    status: PluginInstallTaskStatus = Field(description="The status of the install task.")
+    message: str = Field(description="The message of the install task.")
+    icon: str = Field(description="The icon of the plugin.")
+    labels: I18nObject = Field(description="The labels of the plugin.")
+
+
+class PluginInstallTask(BasePluginEntity):
+    status: PluginInstallTaskStatus = Field(description="The status of the install task.")
+    total_plugins: int = Field(description="The total number of plugins to be installed.")
+    completed_plugins: int = Field(description="The number of plugins that have been installed.")
+    plugins: list[PluginInstallTaskPluginStatus] = Field(description="The status of the plugins.")
+
+
+class PluginInstallTaskStartResponse(BaseModel):
+    all_installed: bool = Field(description="Whether all plugins are installed.")
+    task_id: str = Field(description="The ID of the install task.")
+
+
+class PluginUploadResponse(BaseModel):
+    unique_identifier: str = Field(description="The unique identifier of the plugin.")
+    manifest: PluginDeclaration

api/core/plugin/entities/request.py

@@ -0,0 +1,206 @@
+from typing import Any, Literal, Optional
+
+from pydantic import BaseModel, ConfigDict, Field, field_validator
+
+from core.entities.provider_entities import BasicProviderConfig
+from core.model_runtime.entities.message_entities import (
+    AssistantPromptMessage,
+    PromptMessage,
+    PromptMessageRole,
+    PromptMessageTool,
+    SystemPromptMessage,
+    ToolPromptMessage,
+    UserPromptMessage,
+)
+from core.model_runtime.entities.model_entities import ModelType
+from core.workflow.nodes.parameter_extractor.entities import (
+    ModelConfig as ParameterExtractorModelConfig,
+)
+from core.workflow.nodes.parameter_extractor.entities import (
+    ParameterConfig,
+)
+from core.workflow.nodes.question_classifier.entities import (
+    ClassConfig,
+)
+from core.workflow.nodes.question_classifier.entities import (
+    ModelConfig as QuestionClassifierModelConfig,
+)
+
+
+class RequestInvokeTool(BaseModel):
+    """
+    Request to invoke a tool
+    """
+
+    tool_type: Literal["builtin", "workflow", "api"]
+    provider: str
+    tool: str
+    tool_parameters: dict
+
+
+class BaseRequestInvokeModel(BaseModel):
+    provider: str
+    model: str
+    model_type: ModelType
+
+    model_config = ConfigDict(protected_namespaces=())
+
+
+class RequestInvokeLLM(BaseRequestInvokeModel):
+    """
+    Request to invoke LLM
+    """
+
+    model_type: ModelType = ModelType.LLM
+    mode: str
+    completion_params: dict[str, Any] = Field(default_factory=dict)
+    prompt_messages: list[PromptMessage] = Field(default_factory=list)
+    tools: Optional[list[PromptMessageTool]] = Field(default_factory=list)
+    stop: Optional[list[str]] = Field(default_factory=list)
+    stream: Optional[bool] = False
+
+    model_config = ConfigDict(protected_namespaces=())
+
+    @field_validator("prompt_messages", mode="before")
+    @classmethod
+    def convert_prompt_messages(cls, v):
+        if not isinstance(v, list):
+            raise ValueError("prompt_messages must be a list")
+
+        for i in range(len(v)):
+            if v[i]["role"] == PromptMessageRole.USER.value:
+                v[i] = UserPromptMessage(**v[i])
+            elif v[i]["role"] == PromptMessageRole.ASSISTANT.value:
+                v[i] = AssistantPromptMessage(**v[i])
+            elif v[i]["role"] == PromptMessageRole.SYSTEM.value:
+                v[i] = SystemPromptMessage(**v[i])
+            elif v[i]["role"] == PromptMessageRole.TOOL.value:
+                v[i] = ToolPromptMessage(**v[i])
+            else:
+                v[i] = PromptMessage(**v[i])
+
+        return v
+
+
+class RequestInvokeTextEmbedding(BaseRequestInvokeModel):
+    """
+    Request to invoke text embedding
+    """
+
+    model_type: ModelType = ModelType.TEXT_EMBEDDING
+    texts: list[str]
+
+
+class RequestInvokeRerank(BaseRequestInvokeModel):
+    """
+    Request to invoke rerank
+    """
+
+    model_type: ModelType = ModelType.RERANK
+    query: str
+    docs: list[str]
+    score_threshold: float
+    top_n: int
+
+
+class RequestInvokeTTS(BaseRequestInvokeModel):
+    """
+    Request to invoke TTS
+    """
+
+    model_type: ModelType = ModelType.TTS
+    content_text: str
+    voice: str
+
+
+class RequestInvokeSpeech2Text(BaseRequestInvokeModel):
+    """
+    Request to invoke speech2text
+    """
+
+    model_type: ModelType = ModelType.SPEECH2TEXT
+    file: bytes
+
+    @field_validator("file", mode="before")
+    @classmethod
+    def convert_file(cls, v):
+        # hex string to bytes
+        if isinstance(v, str):
+            return bytes.fromhex(v)
+        else:
+            raise ValueError("file must be a hex string")
+
+
+class RequestInvokeModeration(BaseRequestInvokeModel):
+    """
+    Request to invoke moderation
+    """
+
+    model_type: ModelType = ModelType.MODERATION
+    text: str
+
+
+class RequestInvokeParameterExtractorNode(BaseModel):
+    """
+    Request to invoke parameter extractor node
+    """
+
+    parameters: list[ParameterConfig]
+    model: ParameterExtractorModelConfig
+    instruction: str
+    query: str
+
+
+class RequestInvokeQuestionClassifierNode(BaseModel):
+    """
+    Request to invoke question classifier node
+    """
+
+    query: str
+    model: QuestionClassifierModelConfig
+    classes: list[ClassConfig]
+    instruction: str
+
+
+class RequestInvokeApp(BaseModel):
+    """
+    Request to invoke app
+    """
+
+    app_id: str
+    inputs: dict[str, Any]
+    query: Optional[str] = None
+    response_mode: Literal["blocking", "streaming"]
+    conversation_id: Optional[str] = None
+    user: Optional[str] = None
+    files: list[dict] = Field(default_factory=list)
+
+
+class RequestInvokeEncrypt(BaseModel):
+    """
+    Request to encryption
+    """
+
+    opt: Literal["encrypt", "decrypt", "clear"]
+    namespace: Literal["endpoint"]
+    identity: str
+    data: dict = Field(default_factory=dict)
+    config: list[BasicProviderConfig] = Field(default_factory=list)
+
+
+class RequestInvokeSummary(BaseModel):
+    """
+    Request to summary
+    """
+
+    text: str
+    instruction: str
+
+
+class RequestRequestUploadFile(BaseModel):
+    """
+    Request to upload file
+    """
+
+    filename: str
+    mimetype: str

api/core/plugin/manager/agent.py

@@ -0,0 +1,113 @@
+from collections.abc import Generator
+from typing import Any, Optional
+
+from core.agent.entities import AgentInvokeMessage
+from core.plugin.entities.plugin import GenericProviderID
+from core.plugin.entities.plugin_daemon import (
+    PluginAgentProviderEntity,
+)
+from core.plugin.manager.base import BasePluginManager
+
+
+class PluginAgentManager(BasePluginManager):
+    def fetch_agent_strategy_providers(self, tenant_id: str) -> list[PluginAgentProviderEntity]:
+        """
+        Fetch agent providers for the given tenant.
+        """
+
+        def transformer(json_response: dict[str, Any]) -> dict:
+            for provider in json_response.get("data", []):
+                declaration = provider.get("declaration", {}) or {}
+                provider_name = declaration.get("identity", {}).get("name")
+                for strategy in declaration.get("strategies", []):
+                    strategy["identity"]["provider"] = provider_name
+
+            return json_response
+
+        response = self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/management/agent_strategies",
+            list[PluginAgentProviderEntity],
+            params={"page": 1, "page_size": 256},
+            transformer=transformer,
+        )
+
+        for provider in response:
+            provider.declaration.identity.name = f"{provider.plugin_id}/{provider.declaration.identity.name}"
+
+            # override the provider name for each tool to plugin_id/provider_name
+            for strategy in provider.declaration.strategies:
+                strategy.identity.provider = provider.declaration.identity.name
+
+        return response
+
+    def fetch_agent_strategy_provider(self, tenant_id: str, provider: str) -> PluginAgentProviderEntity:
+        """
+        Fetch tool provider for the given tenant and plugin.
+        """
+        agent_provider_id = GenericProviderID(provider)
+
+        def transformer(json_response: dict[str, Any]) -> dict:
+            # skip if error occurs
+            if json_response.get("data") is None or json_response.get("data", {}).get("declaration") is None:
+                return json_response
+
+            for strategy in json_response.get("data", {}).get("declaration", {}).get("strategies", []):
+                strategy["identity"]["provider"] = agent_provider_id.provider_name
+
+            return json_response
+
+        response = self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/management/agent_strategy",
+            PluginAgentProviderEntity,
+            params={"provider": agent_provider_id.provider_name, "plugin_id": agent_provider_id.plugin_id},
+            transformer=transformer,
+        )
+
+        response.declaration.identity.name = f"{response.plugin_id}/{response.declaration.identity.name}"
+
+        # override the provider name for each tool to plugin_id/provider_name
+        for strategy in response.declaration.strategies:
+            strategy.identity.provider = response.declaration.identity.name
+
+        return response
+
+    def invoke(
+        self,
+        tenant_id: str,
+        user_id: str,
+        agent_provider: str,
+        agent_strategy: str,
+        agent_params: dict[str, Any],
+        conversation_id: Optional[str] = None,
+        app_id: Optional[str] = None,
+        message_id: Optional[str] = None,
+    ) -> Generator[AgentInvokeMessage, None, None]:
+        """
+        Invoke the agent with the given tenant, user, plugin, provider, name and parameters.
+        """
+
+        agent_provider_id = GenericProviderID(agent_provider)
+
+        response = self._request_with_plugin_daemon_response_stream(
+            "POST",
+            f"plugin/{tenant_id}/dispatch/agent_strategy/invoke",
+            AgentInvokeMessage,
+            data={
+                "user_id": user_id,
+                "conversation_id": conversation_id,
+                "app_id": app_id,
+                "message_id": message_id,
+                "data": {
+                    "agent_strategy_provider": agent_provider_id.provider_name,
+                    "agent_strategy": agent_strategy,
+                    "agent_strategy_params": agent_params,
+                },
+            },
+            headers={
+                "X-Plugin-ID": agent_provider_id.plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+        return response

api/core/plugin/manager/asset.py

@@ -0,0 +1,12 @@
+from core.plugin.manager.base import BasePluginManager
+
+
+class PluginAssetManager(BasePluginManager):
+    def fetch_asset(self, tenant_id: str, id: str) -> bytes:
+        """
+        Fetch an asset by id.
+        """
+        response = self._request(method="GET", path=f"plugin/{tenant_id}/asset/{id}")
+        if response.status_code != 200:
+            raise ValueError(f"can not found asset {id}")
+        return response.content

api/core/plugin/manager/base.py

@@ -0,0 +1,237 @@
+import inspect
+import json
+import logging
+from collections.abc import Callable, Generator
+from typing import TypeVar
+
+import requests
+from pydantic import BaseModel
+from yarl import URL
+
+from configs import dify_config
+from core.model_runtime.errors.invoke import (
+    InvokeAuthorizationError,
+    InvokeBadRequestError,
+    InvokeConnectionError,
+    InvokeRateLimitError,
+    InvokeServerUnavailableError,
+)
+from core.model_runtime.errors.validate import CredentialsValidateFailedError
+from core.plugin.entities.plugin_daemon import PluginDaemonBasicResponse, PluginDaemonError, PluginDaemonInnerError
+from core.plugin.manager.exc import (
+    PluginDaemonBadRequestError,
+    PluginDaemonInternalServerError,
+    PluginDaemonNotFoundError,
+    PluginDaemonUnauthorizedError,
+    PluginInvokeError,
+    PluginNotFoundError,
+    PluginPermissionDeniedError,
+    PluginUniqueIdentifierError,
+)
+
+plugin_daemon_inner_api_baseurl = dify_config.PLUGIN_DAEMON_URL
+plugin_daemon_inner_api_key = dify_config.PLUGIN_DAEMON_KEY
+
+T = TypeVar("T", bound=(BaseModel | dict | list | bool | str))
+
+logger = logging.getLogger(__name__)
+
+
+class BasePluginManager:
+    def _request(
+        self,
+        method: str,
+        path: str,
+        headers: dict | None = None,
+        data: bytes | dict | str | None = None,
+        params: dict | None = None,
+        files: dict | None = None,
+        stream: bool = False,
+    ) -> requests.Response:
+        """
+        Make a request to the plugin daemon inner API.
+        """
+        url = URL(str(plugin_daemon_inner_api_baseurl)) / path
+        headers = headers or {}
+        headers["X-Api-Key"] = plugin_daemon_inner_api_key
+        headers["Accept-Encoding"] = "gzip, deflate, br"
+
+        if headers.get("Content-Type") == "application/json" and isinstance(data, dict):
+            data = json.dumps(data)
+
+        try:
+            response = requests.request(
+                method=method, url=str(url), headers=headers, data=data, params=params, stream=stream, files=files
+            )
+        except requests.exceptions.ConnectionError:
+            logger.exception("Request to Plugin Daemon Service failed")
+            raise PluginDaemonInnerError(code=-500, message="Request to Plugin Daemon Service failed")
+
+        return response
+
+    def _stream_request(
+        self,
+        method: str,
+        path: str,
+        params: dict | None = None,
+        headers: dict | None = None,
+        data: bytes | dict | None = None,
+        files: dict | None = None,
+    ) -> Generator[bytes, None, None]:
+        """
+        Make a stream request to the plugin daemon inner API
+        """
+        response = self._request(method, path, headers, data, params, files, stream=True)
+        for line in response.iter_lines():
+            line = line.decode("utf-8").strip()
+            if line.startswith("data:"):
+                line = line[5:].strip()
+            if line:
+                yield line
+
+    def _stream_request_with_model(
+        self,
+        method: str,
+        path: str,
+        type: type[T],
+        headers: dict | None = None,
+        data: bytes | dict | None = None,
+        params: dict | None = None,
+        files: dict | None = None,
+    ) -> Generator[T, None, None]:
+        """
+        Make a stream request to the plugin daemon inner API and yield the response as a model.
+        """
+        for line in self._stream_request(method, path, params, headers, data, files):
+            yield type(**json.loads(line))  # type: ignore
+
+    def _request_with_model(
+        self,
+        method: str,
+        path: str,
+        type: type[T],
+        headers: dict | None = None,
+        data: bytes | None = None,
+        params: dict | None = None,
+        files: dict | None = None,
+    ) -> T:
+        """
+        Make a request to the plugin daemon inner API and return the response as a model.
+        """
+        response = self._request(method, path, headers, data, params, files)
+        return type(**response.json())  # type: ignore
+
+    def _request_with_plugin_daemon_response(
+        self,
+        method: str,
+        path: str,
+        type: type[T],
+        headers: dict | None = None,
+        data: bytes | dict | None = None,
+        params: dict | None = None,
+        files: dict | None = None,
+        transformer: Callable[[dict], dict] | None = None,
+    ) -> T:
+        """
+        Make a request to the plugin daemon inner API and return the response as a model.
+        """
+        response = self._request(method, path, headers, data, params, files)
+        json_response = response.json()
+        if transformer:
+            json_response = transformer(json_response)
+
+        rep = PluginDaemonBasicResponse[type](**json_response)  # type: ignore
+        if rep.code != 0:
+            try:
+                error = PluginDaemonError(**json.loads(rep.message))
+            except Exception:
+                raise ValueError(f"{rep.message}, code: {rep.code}")
+
+            self._handle_plugin_daemon_error(error.error_type, error.message)
+        if rep.data is None:
+            frame = inspect.currentframe()
+            raise ValueError(f"got empty data from plugin daemon: {frame.f_lineno if frame else 'unknown'}")
+
+        return rep.data
+
+    def _request_with_plugin_daemon_response_stream(
+        self,
+        method: str,
+        path: str,
+        type: type[T],
+        headers: dict | None = None,
+        data: bytes | dict | None = None,
+        params: dict | None = None,
+        files: dict | None = None,
+    ) -> Generator[T, None, None]:
+        """
+        Make a stream request to the plugin daemon inner API and yield the response as a model.
+        """
+        for line in self._stream_request(method, path, params, headers, data, files):
+            line_data = None
+            try:
+                line_data = json.loads(line)
+                rep = PluginDaemonBasicResponse[type](**line_data)  # type: ignore
+            except Exception:
+                # TODO modify this when line_data has code and message
+                if line_data and "error" in line_data:
+                    raise ValueError(line_data["error"])
+                else:
+                    raise ValueError(line)
+
+            if rep.code != 0:
+                if rep.code == -500:
+                    try:
+                        error = PluginDaemonError(**json.loads(rep.message))
+                    except Exception:
+                        raise PluginDaemonInnerError(code=rep.code, message=rep.message)
+
+                    self._handle_plugin_daemon_error(error.error_type, error.message)
+                raise ValueError(f"plugin daemon: {rep.message}, code: {rep.code}")
+            if rep.data is None:
+                frame = inspect.currentframe()
+                raise ValueError(f"got empty data from plugin daemon: {frame.f_lineno if frame else 'unknown'}")
+            yield rep.data
+
+    def _handle_plugin_daemon_error(self, error_type: str, message: str):
+        """
+        handle the error from plugin daemon
+        """
+        match error_type:
+            case PluginDaemonInnerError.__name__:
+                raise PluginDaemonInnerError(code=-500, message=message)
+            case PluginInvokeError.__name__:
+                error_object = json.loads(message)
+                invoke_error_type = error_object.get("error_type")
+                args = error_object.get("args")
+                match invoke_error_type:
+                    case InvokeRateLimitError.__name__:
+                        raise InvokeRateLimitError(description=args.get("description"))
+                    case InvokeAuthorizationError.__name__:
+                        raise InvokeAuthorizationError(description=args.get("description"))
+                    case InvokeBadRequestError.__name__:
+                        raise InvokeBadRequestError(description=args.get("description"))
+                    case InvokeConnectionError.__name__:
+                        raise InvokeConnectionError(description=args.get("description"))
+                    case InvokeServerUnavailableError.__name__:
+                        raise InvokeServerUnavailableError(description=args.get("description"))
+                    case CredentialsValidateFailedError.__name__:
+                        raise CredentialsValidateFailedError(error_object.get("message"))
+                    case _:
+                        raise PluginInvokeError(description=message)
+            case PluginDaemonInternalServerError.__name__:
+                raise PluginDaemonInternalServerError(description=message)
+            case PluginDaemonBadRequestError.__name__:
+                raise PluginDaemonBadRequestError(description=message)
+            case PluginDaemonNotFoundError.__name__:
+                raise PluginDaemonNotFoundError(description=message)
+            case PluginUniqueIdentifierError.__name__:
+                raise PluginUniqueIdentifierError(description=message)
+            case PluginNotFoundError.__name__:
+                raise PluginNotFoundError(description=message)
+            case PluginDaemonUnauthorizedError.__name__:
+                raise PluginDaemonUnauthorizedError(description=message)
+            case PluginPermissionDeniedError.__name__:
+                raise PluginPermissionDeniedError(description=message)
+            case _:
+                raise Exception(f"got unknown error from plugin daemon: {error_type}, message: {message}")

api/core/plugin/manager/debugging.py

@@ -0,0 +1,17 @@
+from pydantic import BaseModel
+
+from core.plugin.manager.base import BasePluginManager
+
+
+class PluginDebuggingManager(BasePluginManager):
+    def get_debugging_key(self, tenant_id: str) -> str:
+        """
+        Get the debugging key for the given tenant.
+        """
+
+        class Response(BaseModel):
+            key: str
+
+        response = self._request_with_plugin_daemon_response("POST", f"plugin/{tenant_id}/debugging/key", Response)
+
+        return response.key

api/core/plugin/manager/endpoint.py

@@ -0,0 +1,116 @@
+from core.plugin.entities.endpoint import EndpointEntityWithInstance
+from core.plugin.manager.base import BasePluginManager
+
+
+class PluginEndpointManager(BasePluginManager):
+    def create_endpoint(
+        self, tenant_id: str, user_id: str, plugin_unique_identifier: str, name: str, settings: dict
+    ) -> bool:
+        """
+        Create an endpoint for the given plugin.
+
+        Errors will be raised if any error occurs.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/endpoint/setup",
+            bool,
+            headers={
+                "Content-Type": "application/json",
+            },
+            data={
+                "user_id": user_id,
+                "plugin_unique_identifier": plugin_unique_identifier,
+                "settings": settings,
+                "name": name,
+            },
+        )
+
+    def list_endpoints(self, tenant_id: str, user_id: str, page: int, page_size: int):
+        """
+        List all endpoints for the given tenant and user.
+        """
+        return self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/endpoint/list",
+            list[EndpointEntityWithInstance],
+            params={"page": page, "page_size": page_size},
+        )
+
+    def list_endpoints_for_single_plugin(self, tenant_id: str, user_id: str, plugin_id: str, page: int, page_size: int):
+        """
+        List all endpoints for the given tenant, user and plugin.
+        """
+        return self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/endpoint/list/plugin",
+            list[EndpointEntityWithInstance],
+            params={"plugin_id": plugin_id, "page": page, "page_size": page_size},
+        )
+
+    def update_endpoint(self, tenant_id: str, user_id: str, endpoint_id: str, name: str, settings: dict):
+        """
+        Update the settings of the given endpoint.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/endpoint/update",
+            bool,
+            data={
+                "user_id": user_id,
+                "endpoint_id": endpoint_id,
+                "name": name,
+                "settings": settings,
+            },
+            headers={
+                "Content-Type": "application/json",
+            },
+        )
+
+    def delete_endpoint(self, tenant_id: str, user_id: str, endpoint_id: str):
+        """
+        Delete the given endpoint.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/endpoint/remove",
+            bool,
+            data={
+                "endpoint_id": endpoint_id,
+            },
+            headers={
+                "Content-Type": "application/json",
+            },
+        )
+
+    def enable_endpoint(self, tenant_id: str, user_id: str, endpoint_id: str):
+        """
+        Enable the given endpoint.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/endpoint/enable",
+            bool,
+            data={
+                "endpoint_id": endpoint_id,
+            },
+            headers={
+                "Content-Type": "application/json",
+            },
+        )
+
+    def disable_endpoint(self, tenant_id: str, user_id: str, endpoint_id: str):
+        """
+        Disable the given endpoint.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/endpoint/disable",
+            bool,
+            data={
+                "endpoint_id": endpoint_id,
+            },
+            headers={
+                "Content-Type": "application/json",
+            },
+        )

api/core/plugin/manager/exc.py

@@ -0,0 +1,49 @@
+class PluginDaemonError(Exception):
+    """Base class for all plugin daemon errors."""
+
+    def __init__(self, description: str) -> None:
+        self.description = description
+
+    def __str__(self) -> str:
+        # returns the class name and description
+        return f"{self.__class__.__name__}: {self.description}"
+
+
+class PluginDaemonInternalError(PluginDaemonError):
+    pass
+
+
+class PluginDaemonClientSideError(PluginDaemonError):
+    pass
+
+
+class PluginDaemonInternalServerError(PluginDaemonInternalError):
+    description: str = "Internal Server Error"
+
+
+class PluginDaemonUnauthorizedError(PluginDaemonInternalError):
+    description: str = "Unauthorized"
+
+
+class PluginDaemonNotFoundError(PluginDaemonInternalError):
+    description: str = "Not Found"
+
+
+class PluginDaemonBadRequestError(PluginDaemonClientSideError):
+    description: str = "Bad Request"
+
+
+class PluginInvokeError(PluginDaemonClientSideError):
+    description: str = "Invoke Error"
+
+
+class PluginUniqueIdentifierError(PluginDaemonClientSideError):
+    description: str = "Unique Identifier Error"
+
+
+class PluginNotFoundError(PluginDaemonClientSideError):
+    description: str = "Plugin Not Found"
+
+
+class PluginPermissionDeniedError(PluginDaemonClientSideError):
+    description: str = "Permission Denied"

api/core/plugin/manager/model.py

@@ -0,0 +1,531 @@
+import binascii
+from collections.abc import Generator, Sequence
+from typing import IO, Optional
+
+from core.model_runtime.entities.llm_entities import LLMResultChunk
+from core.model_runtime.entities.message_entities import PromptMessage, PromptMessageTool
+from core.model_runtime.entities.model_entities import AIModelEntity
+from core.model_runtime.entities.rerank_entities import RerankResult
+from core.model_runtime.entities.text_embedding_entities import TextEmbeddingResult
+from core.model_runtime.utils.encoders import jsonable_encoder
+from core.plugin.entities.plugin_daemon import (
+    PluginBasicBooleanResponse,
+    PluginDaemonInnerError,
+    PluginLLMNumTokensResponse,
+    PluginModelProviderEntity,
+    PluginModelSchemaEntity,
+    PluginStringResultResponse,
+    PluginTextEmbeddingNumTokensResponse,
+    PluginVoicesResponse,
+)
+from core.plugin.manager.base import BasePluginManager
+
+
+class PluginModelManager(BasePluginManager):
+    def fetch_model_providers(self, tenant_id: str) -> Sequence[PluginModelProviderEntity]:
+        """
+        Fetch model providers for the given tenant.
+        """
+        response = self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/management/models",
+            list[PluginModelProviderEntity],
+            params={"page": 1, "page_size": 256},
+        )
+        return response
+
+    def get_model_schema(
+        self,
+        tenant_id: str,
+        user_id: str,
+        plugin_id: str,
+        provider: str,
+        model_type: str,
+        model: str,
+        credentials: dict,
+    ) -> AIModelEntity | None:
+        """
+        Get model schema
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            "POST",
+            f"plugin/{tenant_id}/dispatch/model/schema",
+            PluginModelSchemaEntity,
+            data={
+                "user_id": user_id,
+                "data": {
+                    "provider": provider,
+                    "model_type": model_type,
+                    "model": model,
+                    "credentials": credentials,
+                },
+            },
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            return resp.model_schema
+
+        return None
+
+    def validate_provider_credentials(
+        self, tenant_id: str, user_id: str, plugin_id: str, provider: str, credentials: dict
+    ) -> bool:
+        """
+        validate the credentials of the provider
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            "POST",
+            f"plugin/{tenant_id}/dispatch/model/validate_provider_credentials",
+            PluginBasicBooleanResponse,
+            data={
+                "user_id": user_id,
+                "data": {
+                    "provider": provider,
+                    "credentials": credentials,
+                },
+            },
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            if resp.credentials and isinstance(resp.credentials, dict):
+                credentials.update(resp.credentials)
+
+            return resp.result
+
+        return False
+
+    def validate_model_credentials(
+        self,
+        tenant_id: str,
+        user_id: str,
+        plugin_id: str,
+        provider: str,
+        model_type: str,
+        model: str,
+        credentials: dict,
+    ) -> bool:
+        """
+        validate the credentials of the provider
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            "POST",
+            f"plugin/{tenant_id}/dispatch/model/validate_model_credentials",
+            PluginBasicBooleanResponse,
+            data={
+                "user_id": user_id,
+                "data": {
+                    "provider": provider,
+                    "model_type": model_type,
+                    "model": model,
+                    "credentials": credentials,
+                },
+            },
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            if resp.credentials and isinstance(resp.credentials, dict):
+                credentials.update(resp.credentials)
+
+            return resp.result
+
+        return False
+
+    def invoke_llm(
+        self,
+        tenant_id: str,
+        user_id: str,
+        plugin_id: str,
+        provider: str,
+        model: str,
+        credentials: dict,
+        prompt_messages: list[PromptMessage],
+        model_parameters: Optional[dict] = None,
+        tools: Optional[list[PromptMessageTool]] = None,
+        stop: Optional[list[str]] = None,
+        stream: bool = True,
+    ) -> Generator[LLMResultChunk, None, None]:
+        """
+        Invoke llm
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            method="POST",
+            path=f"plugin/{tenant_id}/dispatch/llm/invoke",
+            type=LLMResultChunk,
+            data=jsonable_encoder(
+                {
+                    "user_id": user_id,
+                    "data": {
+                        "provider": provider,
+                        "model_type": "llm",
+                        "model": model,
+                        "credentials": credentials,
+                        "prompt_messages": prompt_messages,
+                        "model_parameters": model_parameters,
+                        "tools": tools,
+                        "stop": stop,
+                        "stream": stream,
+                    },
+                }
+            ),
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        try:
+            yield from response
+        except PluginDaemonInnerError as e:
+            raise ValueError(e.message + str(e.code))
+
+    def get_llm_num_tokens(
+        self,
+        tenant_id: str,
+        user_id: str,
+        plugin_id: str,
+        provider: str,
+        model_type: str,
+        model: str,
+        credentials: dict,
+        prompt_messages: list[PromptMessage],
+        tools: Optional[list[PromptMessageTool]] = None,
+    ) -> int:
+        """
+        Get number of tokens for llm
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            method="POST",
+            path=f"plugin/{tenant_id}/dispatch/llm/num_tokens",
+            type=PluginLLMNumTokensResponse,
+            data=jsonable_encoder(
+                {
+                    "user_id": user_id,
+                    "data": {
+                        "provider": provider,
+                        "model_type": model_type,
+                        "model": model,
+                        "credentials": credentials,
+                        "prompt_messages": prompt_messages,
+                        "tools": tools,
+                    },
+                }
+            ),
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            return resp.num_tokens
+
+        return 0
+
+    def invoke_text_embedding(
+        self,
+        tenant_id: str,
+        user_id: str,
+        plugin_id: str,
+        provider: str,
+        model: str,
+        credentials: dict,
+        texts: list[str],
+        input_type: str,
+    ) -> TextEmbeddingResult:
+        """
+        Invoke text embedding
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            method="POST",
+            path=f"plugin/{tenant_id}/dispatch/text_embedding/invoke",
+            type=TextEmbeddingResult,
+            data=jsonable_encoder(
+                {
+                    "user_id": user_id,
+                    "data": {
+                        "provider": provider,
+                        "model_type": "text-embedding",
+                        "model": model,
+                        "credentials": credentials,
+                        "texts": texts,
+                        "input_type": input_type,
+                    },
+                }
+            ),
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            return resp
+
+        raise ValueError("Failed to invoke text embedding")
+
+    def get_text_embedding_num_tokens(
+        self,
+        tenant_id: str,
+        user_id: str,
+        plugin_id: str,
+        provider: str,
+        model: str,
+        credentials: dict,
+        texts: list[str],
+    ) -> list[int]:
+        """
+        Get number of tokens for text embedding
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            method="POST",
+            path=f"plugin/{tenant_id}/dispatch/text_embedding/num_tokens",
+            type=PluginTextEmbeddingNumTokensResponse,
+            data=jsonable_encoder(
+                {
+                    "user_id": user_id,
+                    "data": {
+                        "provider": provider,
+                        "model_type": "text-embedding",
+                        "model": model,
+                        "credentials": credentials,
+                        "texts": texts,
+                    },
+                }
+            ),
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            return resp.num_tokens
+
+        return []
+
+    def invoke_rerank(
+        self,
+        tenant_id: str,
+        user_id: str,
+        plugin_id: str,
+        provider: str,
+        model: str,
+        credentials: dict,
+        query: str,
+        docs: list[str],
+        score_threshold: Optional[float] = None,
+        top_n: Optional[int] = None,
+    ) -> RerankResult:
+        """
+        Invoke rerank
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            method="POST",
+            path=f"plugin/{tenant_id}/dispatch/rerank/invoke",
+            type=RerankResult,
+            data=jsonable_encoder(
+                {
+                    "user_id": user_id,
+                    "data": {
+                        "provider": provider,
+                        "model_type": "rerank",
+                        "model": model,
+                        "credentials": credentials,
+                        "query": query,
+                        "docs": docs,
+                        "score_threshold": score_threshold,
+                        "top_n": top_n,
+                    },
+                }
+            ),
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            return resp
+
+        raise ValueError("Failed to invoke rerank")
+
+    def invoke_tts(
+        self,
+        tenant_id: str,
+        user_id: str,
+        plugin_id: str,
+        provider: str,
+        model: str,
+        credentials: dict,
+        content_text: str,
+        voice: str,
+    ) -> Generator[bytes, None, None]:
+        """
+        Invoke tts
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            method="POST",
+            path=f"plugin/{tenant_id}/dispatch/tts/invoke",
+            type=PluginStringResultResponse,
+            data=jsonable_encoder(
+                {
+                    "user_id": user_id,
+                    "data": {
+                        "provider": provider,
+                        "model_type": "tts",
+                        "model": model,
+                        "credentials": credentials,
+                        "tenant_id": tenant_id,
+                        "content_text": content_text,
+                        "voice": voice,
+                    },
+                }
+            ),
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        try:
+            for result in response:
+                hex_str = result.result
+                yield binascii.unhexlify(hex_str)
+        except PluginDaemonInnerError as e:
+            raise ValueError(e.message + str(e.code))
+
+    def get_tts_model_voices(
+        self,
+        tenant_id: str,
+        user_id: str,
+        plugin_id: str,
+        provider: str,
+        model: str,
+        credentials: dict,
+        language: Optional[str] = None,
+    ) -> list[dict]:
+        """
+        Get tts model voices
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            method="POST",
+            path=f"plugin/{tenant_id}/dispatch/tts/model/voices",
+            type=PluginVoicesResponse,
+            data=jsonable_encoder(
+                {
+                    "user_id": user_id,
+                    "data": {
+                        "provider": provider,
+                        "model_type": "tts",
+                        "model": model,
+                        "credentials": credentials,
+                        "language": language,
+                    },
+                }
+            ),
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            voices = []
+            for voice in resp.voices:
+                voices.append({"name": voice.name, "value": voice.value})
+
+            return voices
+
+        return []
+
+    def invoke_speech_to_text(
+        self,
+        tenant_id: str,
+        user_id: str,
+        plugin_id: str,
+        provider: str,
+        model: str,
+        credentials: dict,
+        file: IO[bytes],
+    ) -> str:
+        """
+        Invoke speech to text
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            method="POST",
+            path=f"plugin/{tenant_id}/dispatch/speech2text/invoke",
+            type=PluginStringResultResponse,
+            data=jsonable_encoder(
+                {
+                    "user_id": user_id,
+                    "data": {
+                        "provider": provider,
+                        "model_type": "speech2text",
+                        "model": model,
+                        "credentials": credentials,
+                        "file": binascii.hexlify(file.read()).decode(),
+                    },
+                }
+            ),
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            return resp.result
+
+        raise ValueError("Failed to invoke speech to text")
+
+    def invoke_moderation(
+        self,
+        tenant_id: str,
+        user_id: str,
+        plugin_id: str,
+        provider: str,
+        model: str,
+        credentials: dict,
+        text: str,
+    ) -> bool:
+        """
+        Invoke moderation
+        """
+        response = self._request_with_plugin_daemon_response_stream(
+            method="POST",
+            path=f"plugin/{tenant_id}/dispatch/moderation/invoke",
+            type=PluginBasicBooleanResponse,
+            data=jsonable_encoder(
+                {
+                    "user_id": user_id,
+                    "data": {
+                        "provider": provider,
+                        "model_type": "moderation",
+                        "model": model,
+                        "credentials": credentials,
+                        "text": text,
+                    },
+                }
+            ),
+            headers={
+                "X-Plugin-ID": plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            return resp.result
+
+        raise ValueError("Failed to invoke moderation")

api/core/plugin/manager/plugin.py

@@ -0,0 +1,249 @@
+from collections.abc import Sequence
+
+from core.plugin.entities.bundle import PluginBundleDependency
+from core.plugin.entities.plugin import (
+    GenericProviderID,
+    MissingPluginDependency,
+    PluginDeclaration,
+    PluginEntity,
+    PluginInstallation,
+    PluginInstallationSource,
+)
+from core.plugin.entities.plugin_daemon import PluginInstallTask, PluginInstallTaskStartResponse, PluginUploadResponse
+from core.plugin.manager.base import BasePluginManager
+
+
+class PluginInstallationManager(BasePluginManager):
+    def fetch_plugin_by_identifier(
+        self,
+        tenant_id: str,
+        identifier: str,
+    ) -> bool:
+        return self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/management/fetch/identifier",
+            bool,
+            params={"plugin_unique_identifier": identifier},
+        )
+
+    def list_plugins(self, tenant_id: str) -> list[PluginEntity]:
+        return self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/management/list",
+            list[PluginEntity],
+            params={"page": 1, "page_size": 256},
+        )
+
+    def upload_pkg(
+        self,
+        tenant_id: str,
+        pkg: bytes,
+        verify_signature: bool = False,
+    ) -> PluginUploadResponse:
+        """
+        Upload a plugin package and return the plugin unique identifier.
+        """
+        body = {
+            "dify_pkg": ("dify_pkg", pkg, "application/octet-stream"),
+        }
+
+        data = {
+            "verify_signature": "true" if verify_signature else "false",
+        }
+
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/management/install/upload/package",
+            PluginUploadResponse,
+            files=body,
+            data=data,
+        )
+
+    def upload_bundle(
+        self,
+        tenant_id: str,
+        bundle: bytes,
+        verify_signature: bool = False,
+    ) -> Sequence[PluginBundleDependency]:
+        """
+        Upload a plugin bundle and return the dependencies.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/management/install/upload/bundle",
+            list[PluginBundleDependency],
+            files={"dify_bundle": ("dify_bundle", bundle, "application/octet-stream")},
+            data={"verify_signature": "true" if verify_signature else "false"},
+        )
+
+    def install_from_identifiers(
+        self,
+        tenant_id: str,
+        identifiers: Sequence[str],
+        source: PluginInstallationSource,
+        metas: list[dict],
+    ) -> PluginInstallTaskStartResponse:
+        """
+        Install a plugin from an identifier.
+        """
+        # exception will be raised if the request failed
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/management/install/identifiers",
+            PluginInstallTaskStartResponse,
+            data={
+                "plugin_unique_identifiers": identifiers,
+                "source": source,
+                "metas": metas,
+            },
+            headers={"Content-Type": "application/json"},
+        )
+
+    def fetch_plugin_installation_tasks(self, tenant_id: str, page: int, page_size: int) -> Sequence[PluginInstallTask]:
+        """
+        Fetch plugin installation tasks.
+        """
+        return self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/management/install/tasks",
+            list[PluginInstallTask],
+            params={"page": page, "page_size": page_size},
+        )
+
+    def fetch_plugin_installation_task(self, tenant_id: str, task_id: str) -> PluginInstallTask:
+        """
+        Fetch a plugin installation task.
+        """
+        return self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/management/install/tasks/{task_id}",
+            PluginInstallTask,
+        )
+
+    def delete_plugin_installation_task(self, tenant_id: str, task_id: str) -> bool:
+        """
+        Delete a plugin installation task.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/management/install/tasks/{task_id}/delete",
+            bool,
+        )
+
+    def delete_all_plugin_installation_task_items(self, tenant_id: str) -> bool:
+        """
+        Delete all plugin installation task items.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/management/install/tasks/delete_all",
+            bool,
+        )
+
+    def delete_plugin_installation_task_item(self, tenant_id: str, task_id: str, identifier: str) -> bool:
+        """
+        Delete a plugin installation task item.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/management/install/tasks/{task_id}/delete/{identifier}",
+            bool,
+        )
+
+    def fetch_plugin_manifest(self, tenant_id: str, plugin_unique_identifier: str) -> PluginDeclaration:
+        """
+        Fetch a plugin manifest.
+        """
+
+        return self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/management/fetch/manifest",
+            PluginDeclaration,
+            params={"plugin_unique_identifier": plugin_unique_identifier},
+        )
+
+    def fetch_plugin_installation_by_ids(
+        self, tenant_id: str, plugin_ids: Sequence[str]
+    ) -> Sequence[PluginInstallation]:
+        """
+        Fetch plugin installations by ids.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/management/installation/fetch/batch",
+            list[PluginInstallation],
+            data={"plugin_ids": plugin_ids},
+            headers={"Content-Type": "application/json"},
+        )
+
+    def fetch_missing_dependencies(
+        self, tenant_id: str, plugin_unique_identifiers: list[str]
+    ) -> list[MissingPluginDependency]:
+        """
+        Fetch missing dependencies
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/management/installation/missing",
+            list[MissingPluginDependency],
+            data={"plugin_unique_identifiers": plugin_unique_identifiers},
+            headers={"Content-Type": "application/json"},
+        )
+
+    def uninstall(self, tenant_id: str, plugin_installation_id: str) -> bool:
+        """
+        Uninstall a plugin.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/management/uninstall",
+            bool,
+            data={
+                "plugin_installation_id": plugin_installation_id,
+            },
+            headers={"Content-Type": "application/json"},
+        )
+
+    def upgrade_plugin(
+        self,
+        tenant_id: str,
+        original_plugin_unique_identifier: str,
+        new_plugin_unique_identifier: str,
+        source: PluginInstallationSource,
+        meta: dict,
+    ) -> PluginInstallTaskStartResponse:
+        """
+        Upgrade a plugin.
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/management/install/upgrade",
+            PluginInstallTaskStartResponse,
+            data={
+                "original_plugin_unique_identifier": original_plugin_unique_identifier,
+                "new_plugin_unique_identifier": new_plugin_unique_identifier,
+                "source": source,
+                "meta": meta,
+            },
+            headers={"Content-Type": "application/json"},
+        )
+
+    def check_tools_existence(self, tenant_id: str, provider_ids: Sequence[GenericProviderID]) -> Sequence[bool]:
+        """
+        Check if the tools exist
+        """
+        return self._request_with_plugin_daemon_response(
+            "POST",
+            f"plugin/{tenant_id}/management/tools/check_existence",
+            list[bool],
+            data={
+                "provider_ids": [
+                    {
+                        "plugin_id": provider_id.plugin_id,
+                        "provider_name": provider_id.provider_name,
+                    }
+                    for provider_id in provider_ids
+                ]
+            },
+            headers={"Content-Type": "application/json"},
+        )

api/core/plugin/manager/tool.py

@@ -0,0 +1,188 @@
+from collections.abc import Generator
+from typing import Any, Optional
+
+from pydantic import BaseModel
+
+from core.plugin.entities.plugin import GenericProviderID
+from core.plugin.entities.plugin_daemon import PluginBasicBooleanResponse, PluginToolProviderEntity
+from core.plugin.manager.base import BasePluginManager
+from core.tools.entities.tool_entities import ToolInvokeMessage, ToolParameter
+
+
+class PluginToolManager(BasePluginManager):
+    def fetch_tool_providers(self, tenant_id: str) -> list[PluginToolProviderEntity]:
+        """
+        Fetch tool providers for the given tenant.
+        """
+
+        def transformer(json_response: dict[str, Any]) -> dict:
+            for provider in json_response.get("data", []):
+                declaration = provider.get("declaration", {}) or {}
+                provider_name = declaration.get("identity", {}).get("name")
+                for tool in declaration.get("tools", []):
+                    tool["identity"]["provider"] = provider_name
+
+            return json_response
+
+        response = self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/management/tools",
+            list[PluginToolProviderEntity],
+            params={"page": 1, "page_size": 256},
+            transformer=transformer,
+        )
+
+        for provider in response:
+            provider.declaration.identity.name = f"{provider.plugin_id}/{provider.declaration.identity.name}"
+
+            # override the provider name for each tool to plugin_id/provider_name
+            for tool in provider.declaration.tools:
+                tool.identity.provider = provider.declaration.identity.name
+
+        return response
+
+    def fetch_tool_provider(self, tenant_id: str, provider: str) -> PluginToolProviderEntity:
+        """
+        Fetch tool provider for the given tenant and plugin.
+        """
+        tool_provider_id = GenericProviderID(provider)
+
+        def transformer(json_response: dict[str, Any]) -> dict:
+            data = json_response.get("data")
+            if data:
+                for tool in data.get("declaration", {}).get("tools", []):
+                    tool["identity"]["provider"] = tool_provider_id.provider_name
+
+            return json_response
+
+        response = self._request_with_plugin_daemon_response(
+            "GET",
+            f"plugin/{tenant_id}/management/tool",
+            PluginToolProviderEntity,
+            params={"provider": tool_provider_id.provider_name, "plugin_id": tool_provider_id.plugin_id},
+            transformer=transformer,
+        )
+
+        response.declaration.identity.name = f"{response.plugin_id}/{response.declaration.identity.name}"
+
+        # override the provider name for each tool to plugin_id/provider_name
+        for tool in response.declaration.tools:
+            tool.identity.provider = response.declaration.identity.name
+
+        return response
+
+    def invoke(
+        self,
+        tenant_id: str,
+        user_id: str,
+        tool_provider: str,
+        tool_name: str,
+        credentials: dict[str, Any],
+        tool_parameters: dict[str, Any],
+        conversation_id: Optional[str] = None,
+        app_id: Optional[str] = None,
+        message_id: Optional[str] = None,
+    ) -> Generator[ToolInvokeMessage, None, None]:
+        """
+        Invoke the tool with the given tenant, user, plugin, provider, name, credentials and parameters.
+        """
+
+        tool_provider_id = GenericProviderID(tool_provider)
+
+        response = self._request_with_plugin_daemon_response_stream(
+            "POST",
+            f"plugin/{tenant_id}/dispatch/tool/invoke",
+            ToolInvokeMessage,
+            data={
+                "user_id": user_id,
+                "conversation_id": conversation_id,
+                "app_id": app_id,
+                "message_id": message_id,
+                "data": {
+                    "provider": tool_provider_id.provider_name,
+                    "tool": tool_name,
+                    "credentials": credentials,
+                    "tool_parameters": tool_parameters,
+                },
+            },
+            headers={
+                "X-Plugin-ID": tool_provider_id.plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+        return response
+
+    def validate_provider_credentials(
+        self, tenant_id: str, user_id: str, provider: str, credentials: dict[str, Any]
+    ) -> bool:
+        """
+        validate the credentials of the provider
+        """
+        tool_provider_id = GenericProviderID(provider)
+
+        response = self._request_with_plugin_daemon_response_stream(
+            "POST",
+            f"plugin/{tenant_id}/dispatch/tool/validate_credentials",
+            PluginBasicBooleanResponse,
+            data={
+                "user_id": user_id,
+                "data": {
+                    "provider": tool_provider_id.provider_name,
+                    "credentials": credentials,
+                },
+            },
+            headers={
+                "X-Plugin-ID": tool_provider_id.plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            return resp.result
+
+        return False
+
+    def get_runtime_parameters(
+        self,
+        tenant_id: str,
+        user_id: str,
+        provider: str,
+        credentials: dict[str, Any],
+        tool: str,
+        conversation_id: Optional[str] = None,
+        app_id: Optional[str] = None,
+        message_id: Optional[str] = None,
+    ) -> list[ToolParameter]:
+        """
+        get the runtime parameters of the tool
+        """
+        tool_provider_id = GenericProviderID(provider)
+
+        class RuntimeParametersResponse(BaseModel):
+            parameters: list[ToolParameter]
+
+        response = self._request_with_plugin_daemon_response_stream(
+            "POST",
+            f"plugin/{tenant_id}/dispatch/tool/get_runtime_parameters",
+            RuntimeParametersResponse,
+            data={
+                "user_id": user_id,
+                "conversation_id": conversation_id,
+                "app_id": app_id,
+                "message_id": message_id,
+                "data": {
+                    "provider": tool_provider_id.provider_name,
+                    "tool": tool,
+                    "credentials": credentials,
+                },
+            },
+            headers={
+                "X-Plugin-ID": tool_provider_id.plugin_id,
+                "Content-Type": "application/json",
+            },
+        )
+
+        for resp in response:
+            return resp.parameters
+
+        return []

api/core/plugin/utils/converter.py

@@ -0,0 +1,21 @@
+from typing import Any
+
+from core.file.models import File
+from core.tools.entities.tool_entities import ToolSelector
+
+
+def convert_parameters_to_plugin_format(parameters: dict[str, Any]) -> dict[str, Any]:
+    for parameter_name, parameter in parameters.items():
+        if isinstance(parameter, File):
+            parameters[parameter_name] = parameter.to_plugin_parameter()
+        elif isinstance(parameter, list) and all(isinstance(p, File) for p in parameter):
+            parameters[parameter_name] = []
+            for p in parameter:
+                parameters[parameter_name].append(p.to_plugin_parameter())
+        elif isinstance(parameter, ToolSelector):
+            parameters[parameter_name] = parameter.to_plugin_parameter()
+        elif isinstance(parameter, list) and all(isinstance(p, ToolSelector) for p in parameter):
+            parameters[parameter_name] = []
+            for p in parameter:
+                parameters[parameter_name].append(p.to_plugin_parameter())
+    return parameters