Introduce Plugins (#13836)
Signed-off-by: yihong0618 <zouzou0208@gmail.com> Signed-off-by: -LAN- <laipz8200@outlook.com> Signed-off-by: xhe <xw897002528@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: takatost <takatost@gmail.com> Co-authored-by: kurokobo <kuro664@gmail.com> Co-authored-by: Novice Lee <novicelee@NoviPro.local> Co-authored-by: zxhlyh <jasonapring2015@outlook.com> Co-authored-by: AkaraChen <akarachen@outlook.com> Co-authored-by: Yi <yxiaoisme@gmail.com> Co-authored-by: Joel <iamjoel007@gmail.com> Co-authored-by: JzoNg <jzongcode@gmail.com> Co-authored-by: twwu <twwu@dify.ai> Co-authored-by: Hiroshi Fujita <fujita-h@users.noreply.github.com> Co-authored-by: AkaraChen <85140972+AkaraChen@users.noreply.github.com> Co-authored-by: NFish <douxc512@gmail.com> Co-authored-by: Wu Tianwei <30284043+WTW0313@users.noreply.github.com> Co-authored-by: 非法操作 <hjlarry@163.com> Co-authored-by: Novice <857526207@qq.com> Co-authored-by: Hiroki Nagai <82458324+nagaihiroki-git@users.noreply.github.com> Co-authored-by: Gen Sato <52241300+halogen22@users.noreply.github.com> Co-authored-by: eux <euxuuu@gmail.com> Co-authored-by: huangzhuo1949 <167434202+huangzhuo1949@users.noreply.github.com> Co-authored-by: huangzhuo <huangzhuo1@xiaomi.com> Co-authored-by: lotsik <lotsik@mail.ru> Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com> Co-authored-by: nite-knite <nkCoding@gmail.com> Co-authored-by: Jyong <76649700+JohnJyong@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: gakkiyomi <gakkiyomi@aliyun.com> Co-authored-by: CN-P5 <heibai2006@gmail.com> Co-authored-by: CN-P5 <heibai2006@qq.com> Co-authored-by: Chuehnone <1897025+chuehnone@users.noreply.github.com> Co-authored-by: yihong <zouzou0208@gmail.com> Co-authored-by: Kevin9703 <51311316+Kevin9703@users.noreply.github.com> Co-authored-by: -LAN- <laipz8200@outlook.com> Co-authored-by: Boris Feld <lothiraldan@gmail.com> Co-authored-by: mbo <himabo@gmail.com> Co-authored-by: mabo <mabo@aeyes.ai> Co-authored-by: Warren Chen <warren.chen830@gmail.com> Co-authored-by: JzoNgKVO <27049666+JzoNgKVO@users.noreply.github.com> Co-authored-by: jiandanfeng <chenjh3@wangsu.com> Co-authored-by: zhu-an <70234959+xhdd123321@users.noreply.github.com> Co-authored-by: zhaoqingyu.1075 <zhaoqingyu.1075@bytedance.com> Co-authored-by: 海狸大師 <86974027+yenslife@users.noreply.github.com> Co-authored-by: Xu Song <xusong.vip@gmail.com> Co-authored-by: rayshaw001 <396301947@163.com> Co-authored-by: Ding Jiatong <dingjiatong@gmail.com> Co-authored-by: Bowen Liang <liangbowen@gf.com.cn> Co-authored-by: JasonVV <jasonwangiii@outlook.com> Co-authored-by: le0zh <newlight@qq.com> Co-authored-by: zhuxinliang <zhuxinliang@didiglobal.com> Co-authored-by: k-zaku <zaku99@outlook.jp> Co-authored-by: luckylhb90 <luckylhb90@gmail.com> Co-authored-by: hobo.l <hobo.l@binance.com> Co-authored-by: jiangbo721 <365065261@qq.com> Co-authored-by: 刘江波 <jiangbo721@163.com> Co-authored-by: Shun Miyazawa <34241526+miya@users.noreply.github.com> Co-authored-by: EricPan <30651140+Egfly@users.noreply.github.com> Co-authored-by: crazywoola <427733928@qq.com> Co-authored-by: sino <sino2322@gmail.com> Co-authored-by: Jhvcc <37662342+Jhvcc@users.noreply.github.com> Co-authored-by: lowell <lowell.hu@zkteco.in> Co-authored-by: Boris Polonsky <BorisPolonsky@users.noreply.github.com> Co-authored-by: Ademílson Tonato <ademilsonft@outlook.com> Co-authored-by: Ademílson Tonato <ademilson.tonato@refurbed.com> Co-authored-by: IWAI, Masaharu <iwaim.sub@gmail.com> Co-authored-by: Yueh-Po Peng (Yabi) <94939112+y10ab1@users.noreply.github.com> Co-authored-by: Jason <ggbbddjm@gmail.com> Co-authored-by: Xin Zhang <sjhpzx@gmail.com> Co-authored-by: yjc980121 <3898524+yjc980121@users.noreply.github.com> Co-authored-by: heyszt <36215648+hieheihei@users.noreply.github.com> Co-authored-by: Abdullah AlOsaimi <osaimiacc@gmail.com> Co-authored-by: Abdullah AlOsaimi <189027247+osaimi@users.noreply.github.com> Co-authored-by: Yingchun Lai <laiyingchun@apache.org> Co-authored-by: Hash Brown <hi@xzd.me> Co-authored-by: zuodongxu <192560071+zuodongxu@users.noreply.github.com> Co-authored-by: Masashi Tomooka <tmokmss@users.noreply.github.com> Co-authored-by: aplio <ryo.091219@gmail.com> Co-authored-by: Obada Khalili <54270856+obadakhalili@users.noreply.github.com> Co-authored-by: Nam Vu <zuzoovn@gmail.com> Co-authored-by: Kei YAMAZAKI <1715090+kei-yamazaki@users.noreply.github.com> Co-authored-by: TechnoHouse <13776377+deephbz@users.noreply.github.com> Co-authored-by: Riddhimaan-Senapati <114703025+Riddhimaan-Senapati@users.noreply.github.com> Co-authored-by: MaFee921 <31881301+2284730142@users.noreply.github.com> Co-authored-by: te-chan <t-nakanome@sakura-is.co.jp> Co-authored-by: HQidea <HQidea@users.noreply.github.com> Co-authored-by: Joshbly <36315710+Joshbly@users.noreply.github.com> Co-authored-by: xhe <xw897002528@gmail.com> Co-authored-by: weiwenyan-dev <154779315+weiwenyan-dev@users.noreply.github.com> Co-authored-by: ex_wenyan.wei <ex_wenyan.wei@tcl.com> Co-authored-by: engchina <12236799+engchina@users.noreply.github.com> Co-authored-by: engchina <atjapan2015@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: 呆萌闷油瓶 <253605712@qq.com> Co-authored-by: Kemal <kemalmeler@outlook.com> Co-authored-by: Lazy_Frog <4590648+lazyFrogLOL@users.noreply.github.com> Co-authored-by: Yi Xiao <54782454+YIXIAO0@users.noreply.github.com> Co-authored-by: Steven sun <98230804+Tuyohai@users.noreply.github.com> Co-authored-by: steven <sunzwj@digitalchina.com> Co-authored-by: Kalo Chin <91766386+fdb02983rhy@users.noreply.github.com> Co-authored-by: Katy Tao <34019945+KatyTao@users.noreply.github.com> Co-authored-by: depy <42985524+h4ckdepy@users.noreply.github.com> Co-authored-by: 胡春东 <gycm520@gmail.com> Co-authored-by: Junjie.M <118170653@qq.com> Co-authored-by: MuYu <mr.muzea@gmail.com> Co-authored-by: Naoki Takashima <39912547+takatea@users.noreply.github.com> Co-authored-by: Summer-Gu <37869445+gubinjie@users.noreply.github.com> Co-authored-by: Fei He <droxer.he@gmail.com> Co-authored-by: ybalbert001 <120714773+ybalbert001@users.noreply.github.com> Co-authored-by: Yuanbo Li <ybalbert@amazon.com> Co-authored-by: douxc <7553076+douxc@users.noreply.github.com> Co-authored-by: liuzhenghua <1090179900@qq.com> Co-authored-by: Wu Jiayang <62842862+Wu-Jiayang@users.noreply.github.com> Co-authored-by: Your Name <you@example.com> Co-authored-by: kimjion <45935338+kimjion@users.noreply.github.com> Co-authored-by: AugNSo <song.tiankai@icloud.com> Co-authored-by: llinvokerl <38915183+llinvokerl@users.noreply.github.com> Co-authored-by: liusurong.lsr <liusurong.lsr@alibaba-inc.com> Co-authored-by: Vasu Negi <vasu-negi@users.noreply.github.com> Co-authored-by: Hundredwz <1808096180@qq.com> Co-authored-by: Xiyuan Chen <52963600+GareArc@users.noreply.github.com>
This commit is contained in:
Yeuoly
9
api/core/plugin/entities/base.py
Normal file
9
api/core/plugin/entities/base.py
Normal file
@@ -0,0 +1,9 @@
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class BasePluginEntity(BaseModel):
|
||||
id: str
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
30
api/core/plugin/entities/bundle.py
Normal file
30
api/core/plugin/entities/bundle.py
Normal file
@@ -0,0 +1,30 @@
|
||||
from enum import StrEnum
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from core.plugin.entities.plugin import PluginDeclaration, PluginInstallationSource
|
||||
|
||||
|
||||
class PluginBundleDependency(BaseModel):
|
||||
class Type(StrEnum):
|
||||
Github = PluginInstallationSource.Github.value
|
||||
Marketplace = PluginInstallationSource.Marketplace.value
|
||||
Package = PluginInstallationSource.Package.value
|
||||
|
||||
class Github(BaseModel):
|
||||
repo_address: str
|
||||
repo: str
|
||||
release: str
|
||||
packages: str
|
||||
|
||||
class Marketplace(BaseModel):
|
||||
organization: str
|
||||
plugin: str
|
||||
version: str
|
||||
|
||||
class Package(BaseModel):
|
||||
unique_identifier: str
|
||||
manifest: PluginDeclaration
|
||||
|
||||
type: Type
|
||||
value: Github | Marketplace | Package
|
||||
54
api/core/plugin/entities/endpoint.py
Normal file
54
api/core/plugin/entities/endpoint.py
Normal file
@@ -0,0 +1,54 @@
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
|
||||
from configs import dify_config
|
||||
from core.entities.provider_entities import ProviderConfig
|
||||
from core.plugin.entities.base import BasePluginEntity
|
||||
|
||||
|
||||
class EndpointDeclaration(BaseModel):
|
||||
"""
|
||||
declaration of an endpoint
|
||||
"""
|
||||
|
||||
path: str
|
||||
method: str
|
||||
hidden: bool = Field(default=False)
|
||||
|
||||
|
||||
class EndpointProviderDeclaration(BaseModel):
|
||||
"""
|
||||
declaration of an endpoint group
|
||||
"""
|
||||
|
||||
settings: list[ProviderConfig] = Field(default_factory=list)
|
||||
endpoints: Optional[list[EndpointDeclaration]] = Field(default_factory=list)
|
||||
|
||||
|
||||
class EndpointEntity(BasePluginEntity):
|
||||
"""
|
||||
entity of an endpoint
|
||||
"""
|
||||
|
||||
settings: dict
|
||||
tenant_id: str
|
||||
plugin_id: str
|
||||
expired_at: datetime
|
||||
declaration: EndpointProviderDeclaration = Field(default_factory=EndpointProviderDeclaration)
|
||||
|
||||
|
||||
class EndpointEntityWithInstance(EndpointEntity):
|
||||
name: str
|
||||
enabled: bool
|
||||
url: str
|
||||
hook_id: str
|
||||
|
||||
@model_validator(mode="before")
|
||||
@classmethod
|
||||
def render_url_template(cls, values):
|
||||
if "url" not in values:
|
||||
url_template = dify_config.ENDPOINT_URL_TEMPLATE
|
||||
values["url"] = url_template.replace("{hook_id}", values["hook_id"])
|
||||
return values
|
||||
45
api/core/plugin/entities/marketplace.py
Normal file
45
api/core/plugin/entities/marketplace.py
Normal file
@@ -0,0 +1,45 @@
|
||||
from typing import Optional
|
||||
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
|
||||
from core.model_runtime.entities.provider_entities import ProviderEntity
|
||||
from core.plugin.entities.endpoint import EndpointProviderDeclaration
|
||||
from core.plugin.entities.plugin import PluginResourceRequirements
|
||||
from core.tools.entities.common_entities import I18nObject
|
||||
from core.tools.entities.tool_entities import ToolProviderEntity
|
||||
|
||||
|
||||
class MarketplacePluginDeclaration(BaseModel):
|
||||
name: str = Field(..., description="Unique identifier for the plugin within the marketplace")
|
||||
org: str = Field(..., description="Organization or developer responsible for creating and maintaining the plugin")
|
||||
plugin_id: str = Field(..., description="Globally unique identifier for the plugin across all marketplaces")
|
||||
icon: str = Field(..., description="URL or path to the plugin's visual representation")
|
||||
label: I18nObject = Field(..., description="Localized display name for the plugin in different languages")
|
||||
brief: I18nObject = Field(..., description="Short, localized description of the plugin's functionality")
|
||||
resource: PluginResourceRequirements = Field(
|
||||
..., description="Specification of computational resources needed to run the plugin"
|
||||
)
|
||||
endpoint: Optional[EndpointProviderDeclaration] = Field(
|
||||
None, description="Configuration for the plugin's API endpoint, if applicable"
|
||||
)
|
||||
model: Optional[ProviderEntity] = Field(None, description="Details of the AI model used by the plugin, if any")
|
||||
tool: Optional[ToolProviderEntity] = Field(
|
||||
None, description="Information about the tool functionality provided by the plugin, if any"
|
||||
)
|
||||
latest_version: str = Field(
|
||||
..., description="Most recent version number of the plugin available in the marketplace"
|
||||
)
|
||||
latest_package_identifier: str = Field(
|
||||
..., description="Unique identifier for the latest package release of the plugin"
|
||||
)
|
||||
|
||||
@model_validator(mode="before")
|
||||
@classmethod
|
||||
def transform_declaration(cls, data: dict):
|
||||
if "endpoint" in data and not data["endpoint"]:
|
||||
del data["endpoint"]
|
||||
if "model" in data and not data["model"]:
|
||||
del data["model"]
|
||||
if "tool" in data and not data["tool"]:
|
||||
del data["tool"]
|
||||
return data
|
||||
162
api/core/plugin/entities/parameters.py
Normal file
162
api/core/plugin/entities/parameters.py
Normal file
@@ -0,0 +1,162 @@
|
||||
import enum
|
||||
from typing import Any, Optional, Union
|
||||
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from core.entities.parameter_entities import CommonParameterType
|
||||
from core.tools.entities.common_entities import I18nObject
|
||||
|
||||
|
||||
class PluginParameterOption(BaseModel):
|
||||
value: str = Field(..., description="The value of the option")
|
||||
label: I18nObject = Field(..., description="The label of the option")
|
||||
|
||||
@field_validator("value", mode="before")
|
||||
@classmethod
|
||||
def transform_id_to_str(cls, value) -> str:
|
||||
if not isinstance(value, str):
|
||||
return str(value)
|
||||
else:
|
||||
return value
|
||||
|
||||
|
||||
class PluginParameterType(enum.StrEnum):
|
||||
"""
|
||||
all available parameter types
|
||||
"""
|
||||
|
||||
STRING = CommonParameterType.STRING.value
|
||||
NUMBER = CommonParameterType.NUMBER.value
|
||||
BOOLEAN = CommonParameterType.BOOLEAN.value
|
||||
SELECT = CommonParameterType.SELECT.value
|
||||
SECRET_INPUT = CommonParameterType.SECRET_INPUT.value
|
||||
FILE = CommonParameterType.FILE.value
|
||||
FILES = CommonParameterType.FILES.value
|
||||
APP_SELECTOR = CommonParameterType.APP_SELECTOR.value
|
||||
MODEL_SELECTOR = CommonParameterType.MODEL_SELECTOR.value
|
||||
TOOLS_SELECTOR = CommonParameterType.TOOLS_SELECTOR.value
|
||||
|
||||
# deprecated, should not use.
|
||||
SYSTEM_FILES = CommonParameterType.SYSTEM_FILES.value
|
||||
|
||||
|
||||
class PluginParameterAutoGenerate(BaseModel):
|
||||
class Type(enum.StrEnum):
|
||||
PROMPT_INSTRUCTION = "prompt_instruction"
|
||||
|
||||
type: Type
|
||||
|
||||
|
||||
class PluginParameterTemplate(BaseModel):
|
||||
enabled: bool = Field(default=False, description="Whether the parameter is jinja enabled")
|
||||
|
||||
|
||||
class PluginParameter(BaseModel):
|
||||
name: str = Field(..., description="The name of the parameter")
|
||||
label: I18nObject = Field(..., description="The label presented to the user")
|
||||
placeholder: Optional[I18nObject] = Field(default=None, description="The placeholder presented to the user")
|
||||
scope: str | None = None
|
||||
auto_generate: Optional[PluginParameterAutoGenerate] = None
|
||||
template: Optional[PluginParameterTemplate] = None
|
||||
required: bool = False
|
||||
default: Optional[Union[float, int, str]] = None
|
||||
min: Optional[Union[float, int]] = None
|
||||
max: Optional[Union[float, int]] = None
|
||||
precision: Optional[int] = None
|
||||
options: list[PluginParameterOption] = Field(default_factory=list)
|
||||
|
||||
@field_validator("options", mode="before")
|
||||
@classmethod
|
||||
def transform_options(cls, v):
|
||||
if not isinstance(v, list):
|
||||
return []
|
||||
return v
|
||||
|
||||
|
||||
def as_normal_type(typ: enum.StrEnum):
|
||||
if typ.value in {
|
||||
PluginParameterType.SECRET_INPUT,
|
||||
PluginParameterType.SELECT,
|
||||
}:
|
||||
return "string"
|
||||
return typ.value
|
||||
|
||||
|
||||
def cast_parameter_value(typ: enum.StrEnum, value: Any, /):
|
||||
try:
|
||||
match typ.value:
|
||||
case PluginParameterType.STRING | PluginParameterType.SECRET_INPUT | PluginParameterType.SELECT:
|
||||
if value is None:
|
||||
return ""
|
||||
else:
|
||||
return value if isinstance(value, str) else str(value)
|
||||
|
||||
case PluginParameterType.BOOLEAN:
|
||||
if value is None:
|
||||
return False
|
||||
elif isinstance(value, str):
|
||||
# Allowed YAML boolean value strings: https://yaml.org/type/bool.html
|
||||
# and also '0' for False and '1' for True
|
||||
match value.lower():
|
||||
case "true" | "yes" | "y" | "1":
|
||||
return True
|
||||
case "false" | "no" | "n" | "0":
|
||||
return False
|
||||
case _:
|
||||
return bool(value)
|
||||
else:
|
||||
return value if isinstance(value, bool) else bool(value)
|
||||
|
||||
case PluginParameterType.NUMBER:
|
||||
if isinstance(value, int | float):
|
||||
return value
|
||||
elif isinstance(value, str) and value:
|
||||
if "." in value:
|
||||
return float(value)
|
||||
else:
|
||||
return int(value)
|
||||
case PluginParameterType.SYSTEM_FILES | PluginParameterType.FILES:
|
||||
if not isinstance(value, list):
|
||||
return [value]
|
||||
return value
|
||||
case PluginParameterType.FILE:
|
||||
if isinstance(value, list):
|
||||
if len(value) != 1:
|
||||
raise ValueError("This parameter only accepts one file but got multiple files while invoking.")
|
||||
else:
|
||||
return value[0]
|
||||
return value
|
||||
case PluginParameterType.MODEL_SELECTOR | PluginParameterType.APP_SELECTOR:
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError("The selector must be a dictionary.")
|
||||
return value
|
||||
case PluginParameterType.TOOLS_SELECTOR:
|
||||
if not isinstance(value, list):
|
||||
raise ValueError("The tools selector must be a list.")
|
||||
return value
|
||||
case _:
|
||||
return str(value)
|
||||
except ValueError:
|
||||
raise
|
||||
except Exception:
|
||||
raise ValueError(f"The tool parameter value {value} is not in correct type of {as_normal_type(typ)}.")
|
||||
|
||||
|
||||
def init_frontend_parameter(rule: PluginParameter, type: enum.StrEnum, value: Any):
|
||||
"""
|
||||
init frontend parameter by rule
|
||||
"""
|
||||
parameter_value = value
|
||||
if not parameter_value and parameter_value != 0:
|
||||
# get default value
|
||||
parameter_value = rule.default
|
||||
if not parameter_value and rule.required:
|
||||
raise ValueError(f"tool parameter {rule.name} not found in tool config")
|
||||
|
||||
if type == PluginParameterType.SELECT:
|
||||
# check if tool_parameter_config in options
|
||||
options = [x.value for x in rule.options]
|
||||
if parameter_value is not None and parameter_value not in options:
|
||||
raise ValueError(f"tool parameter {rule.name} value {parameter_value} not in options {options}")
|
||||
|
||||
return cast_parameter_value(type, parameter_value)
|
||||
220
api/core/plugin/entities/plugin.py
Normal file
220
api/core/plugin/entities/plugin.py
Normal file
@@ -0,0 +1,220 @@
|
||||
import datetime
|
||||
import enum
|
||||
import re
|
||||
from collections.abc import Mapping
|
||||
from typing import Any, Optional
|
||||
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
|
||||
from core.agent.plugin_entities import AgentStrategyProviderEntity
|
||||
from core.model_runtime.entities.provider_entities import ProviderEntity
|
||||
from core.plugin.entities.base import BasePluginEntity
|
||||
from core.plugin.entities.endpoint import EndpointProviderDeclaration
|
||||
from core.tools.entities.common_entities import I18nObject
|
||||
from core.tools.entities.tool_entities import ToolProviderEntity
|
||||
|
||||
|
||||
class PluginInstallationSource(enum.StrEnum):
|
||||
Github = "github"
|
||||
Marketplace = "marketplace"
|
||||
Package = "package"
|
||||
Remote = "remote"
|
||||
|
||||
|
||||
class PluginResourceRequirements(BaseModel):
|
||||
memory: int
|
||||
|
||||
class Permission(BaseModel):
|
||||
class Tool(BaseModel):
|
||||
enabled: Optional[bool] = Field(default=False)
|
||||
|
||||
class Model(BaseModel):
|
||||
enabled: Optional[bool] = Field(default=False)
|
||||
llm: Optional[bool] = Field(default=False)
|
||||
text_embedding: Optional[bool] = Field(default=False)
|
||||
rerank: Optional[bool] = Field(default=False)
|
||||
tts: Optional[bool] = Field(default=False)
|
||||
speech2text: Optional[bool] = Field(default=False)
|
||||
moderation: Optional[bool] = Field(default=False)
|
||||
|
||||
class Node(BaseModel):
|
||||
enabled: Optional[bool] = Field(default=False)
|
||||
|
||||
class Endpoint(BaseModel):
|
||||
enabled: Optional[bool] = Field(default=False)
|
||||
|
||||
class Storage(BaseModel):
|
||||
enabled: Optional[bool] = Field(default=False)
|
||||
size: int = Field(ge=1024, le=1073741824, default=1048576)
|
||||
|
||||
tool: Optional[Tool] = Field(default=None)
|
||||
model: Optional[Model] = Field(default=None)
|
||||
node: Optional[Node] = Field(default=None)
|
||||
endpoint: Optional[Endpoint] = Field(default=None)
|
||||
storage: Storage = Field(default=None)
|
||||
|
||||
permission: Optional[Permission] = Field(default=None)
|
||||
|
||||
|
||||
class PluginCategory(enum.StrEnum):
|
||||
Tool = "tool"
|
||||
Model = "model"
|
||||
Extension = "extension"
|
||||
AgentStrategy = "agent-strategy"
|
||||
|
||||
|
||||
class PluginDeclaration(BaseModel):
|
||||
class Plugins(BaseModel):
|
||||
tools: Optional[list[str]] = Field(default_factory=list)
|
||||
models: Optional[list[str]] = Field(default_factory=list)
|
||||
endpoints: Optional[list[str]] = Field(default_factory=list)
|
||||
|
||||
version: str = Field(..., pattern=r"^\d{1,4}(\.\d{1,4}){1,3}(-\w{1,16})?$")
|
||||
author: Optional[str] = Field(..., pattern=r"^[a-zA-Z0-9_-]{1,64}$")
|
||||
name: str = Field(..., pattern=r"^[a-z0-9_-]{1,128}$")
|
||||
description: I18nObject
|
||||
icon: str
|
||||
label: I18nObject
|
||||
category: PluginCategory
|
||||
created_at: datetime.datetime
|
||||
resource: PluginResourceRequirements
|
||||
plugins: Plugins
|
||||
tags: list[str] = Field(default_factory=list)
|
||||
verified: bool = Field(default=False)
|
||||
tool: Optional[ToolProviderEntity] = None
|
||||
model: Optional[ProviderEntity] = None
|
||||
endpoint: Optional[EndpointProviderDeclaration] = None
|
||||
agent_strategy: Optional[AgentStrategyProviderEntity] = None
|
||||
|
||||
@model_validator(mode="before")
|
||||
@classmethod
|
||||
def validate_category(cls, values: dict) -> dict:
|
||||
# auto detect category
|
||||
if values.get("tool"):
|
||||
values["category"] = PluginCategory.Tool
|
||||
elif values.get("model"):
|
||||
values["category"] = PluginCategory.Model
|
||||
elif values.get("agent_strategy"):
|
||||
values["category"] = PluginCategory.AgentStrategy
|
||||
else:
|
||||
values["category"] = PluginCategory.Extension
|
||||
return values
|
||||
|
||||
|
||||
class PluginInstallation(BasePluginEntity):
|
||||
tenant_id: str
|
||||
endpoints_setups: int
|
||||
endpoints_active: int
|
||||
runtime_type: str
|
||||
source: PluginInstallationSource
|
||||
meta: Mapping[str, Any]
|
||||
plugin_id: str
|
||||
plugin_unique_identifier: str
|
||||
version: str
|
||||
checksum: str
|
||||
declaration: PluginDeclaration
|
||||
|
||||
|
||||
class PluginEntity(PluginInstallation):
|
||||
name: str
|
||||
installation_id: str
|
||||
version: str
|
||||
latest_version: Optional[str] = None
|
||||
latest_unique_identifier: Optional[str] = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def set_plugin_id(self):
|
||||
if self.declaration.tool:
|
||||
self.declaration.tool.plugin_id = self.plugin_id
|
||||
return self
|
||||
|
||||
|
||||
class GithubPackage(BaseModel):
|
||||
repo: str
|
||||
version: str
|
||||
package: str
|
||||
|
||||
|
||||
class GithubVersion(BaseModel):
|
||||
repo: str
|
||||
version: str
|
||||
|
||||
|
||||
class GenericProviderID:
|
||||
organization: str
|
||||
plugin_name: str
|
||||
provider_name: str
|
||||
is_hardcoded: bool
|
||||
|
||||
def to_string(self) -> str:
|
||||
return str(self)
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f"{self.organization}/{self.plugin_name}/{self.provider_name}"
|
||||
|
||||
def __init__(self, value: str, is_hardcoded: bool = False) -> None:
|
||||
# check if the value is a valid plugin id with format: $organization/$plugin_name/$provider_name
|
||||
if not re.match(r"^[a-z0-9_-]+\/[a-z0-9_-]+\/[a-z0-9_-]+$", value):
|
||||
# check if matches [a-z0-9_-]+, if yes, append with langgenius/$value/$value
|
||||
if re.match(r"^[a-z0-9_-]+$", value):
|
||||
value = f"langgenius/{value}/{value}"
|
||||
else:
|
||||
raise ValueError(f"Invalid plugin id {value}")
|
||||
|
||||
self.organization, self.plugin_name, self.provider_name = value.split("/")
|
||||
self.is_hardcoded = is_hardcoded
|
||||
|
||||
@property
|
||||
def plugin_id(self) -> str:
|
||||
return f"{self.organization}/{self.plugin_name}"
|
||||
|
||||
|
||||
class ModelProviderID(GenericProviderID):
|
||||
def __init__(self, value: str, is_hardcoded: bool = False) -> None:
|
||||
super().__init__(value, is_hardcoded)
|
||||
if self.organization == "langgenius" and self.provider_name == "google":
|
||||
self.plugin_name = "gemini"
|
||||
|
||||
|
||||
class ToolProviderID(GenericProviderID):
|
||||
def __init__(self, value: str, is_hardcoded: bool = False) -> None:
|
||||
super().__init__(value, is_hardcoded)
|
||||
if self.organization == "langgenius":
|
||||
if self.provider_name in ["jina", "siliconflow"]:
|
||||
self.plugin_name = f"{self.provider_name}_tool"
|
||||
|
||||
|
||||
class PluginDependency(BaseModel):
|
||||
class Type(enum.StrEnum):
|
||||
Github = PluginInstallationSource.Github.value
|
||||
Marketplace = PluginInstallationSource.Marketplace.value
|
||||
Package = PluginInstallationSource.Package.value
|
||||
|
||||
class Github(BaseModel):
|
||||
repo: str
|
||||
version: str
|
||||
package: str
|
||||
github_plugin_unique_identifier: str
|
||||
|
||||
@property
|
||||
def plugin_unique_identifier(self) -> str:
|
||||
return self.github_plugin_unique_identifier
|
||||
|
||||
class Marketplace(BaseModel):
|
||||
marketplace_plugin_unique_identifier: str
|
||||
|
||||
@property
|
||||
def plugin_unique_identifier(self) -> str:
|
||||
return self.marketplace_plugin_unique_identifier
|
||||
|
||||
class Package(BaseModel):
|
||||
plugin_unique_identifier: str
|
||||
|
||||
type: Type
|
||||
value: Github | Marketplace | Package
|
||||
current_identifier: Optional[str] = None
|
||||
|
||||
|
||||
class MissingPluginDependency(BaseModel):
|
||||
plugin_unique_identifier: str
|
||||
current_identifier: Optional[str] = None
|
||||
160
api/core/plugin/entities/plugin_daemon.py
Normal file
160
api/core/plugin/entities/plugin_daemon.py
Normal file
@@ -0,0 +1,160 @@
|
||||
from datetime import datetime
|
||||
from enum import StrEnum
|
||||
from typing import Generic, Optional, TypeVar
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from core.agent.plugin_entities import AgentProviderEntityWithPlugin
|
||||
from core.model_runtime.entities.model_entities import AIModelEntity
|
||||
from core.model_runtime.entities.provider_entities import ProviderEntity
|
||||
from core.plugin.entities.base import BasePluginEntity
|
||||
from core.plugin.entities.plugin import PluginDeclaration
|
||||
from core.tools.entities.common_entities import I18nObject
|
||||
from core.tools.entities.tool_entities import ToolProviderEntityWithPlugin
|
||||
|
||||
T = TypeVar("T", bound=(BaseModel | dict | list | bool | str))
|
||||
|
||||
|
||||
class PluginDaemonBasicResponse(BaseModel, Generic[T]):
|
||||
"""
|
||||
Basic response from plugin daemon.
|
||||
"""
|
||||
|
||||
code: int
|
||||
message: str
|
||||
data: Optional[T]
|
||||
|
||||
|
||||
class InstallPluginMessage(BaseModel):
|
||||
"""
|
||||
Message for installing a plugin.
|
||||
"""
|
||||
|
||||
class Event(StrEnum):
|
||||
Info = "info"
|
||||
Done = "done"
|
||||
Error = "error"
|
||||
|
||||
event: Event
|
||||
data: str
|
||||
|
||||
|
||||
class PluginToolProviderEntity(BaseModel):
|
||||
provider: str
|
||||
plugin_unique_identifier: str
|
||||
plugin_id: str
|
||||
declaration: ToolProviderEntityWithPlugin
|
||||
|
||||
|
||||
class PluginAgentProviderEntity(BaseModel):
|
||||
provider: str
|
||||
plugin_unique_identifier: str
|
||||
plugin_id: str
|
||||
declaration: AgentProviderEntityWithPlugin
|
||||
|
||||
|
||||
class PluginBasicBooleanResponse(BaseModel):
|
||||
"""
|
||||
Basic boolean response from plugin daemon.
|
||||
"""
|
||||
|
||||
result: bool
|
||||
credentials: dict | None = None
|
||||
|
||||
|
||||
class PluginModelSchemaEntity(BaseModel):
|
||||
model_schema: AIModelEntity = Field(description="The model schema.")
|
||||
|
||||
# pydantic configs
|
||||
model_config = ConfigDict(protected_namespaces=())
|
||||
|
||||
|
||||
class PluginModelProviderEntity(BaseModel):
|
||||
id: str = Field(description="ID")
|
||||
created_at: datetime = Field(description="The created at time of the model provider.")
|
||||
updated_at: datetime = Field(description="The updated at time of the model provider.")
|
||||
provider: str = Field(description="The provider of the model.")
|
||||
tenant_id: str = Field(description="The tenant ID.")
|
||||
plugin_unique_identifier: str = Field(description="The plugin unique identifier.")
|
||||
plugin_id: str = Field(description="The plugin ID.")
|
||||
declaration: ProviderEntity = Field(description="The declaration of the model provider.")
|
||||
|
||||
|
||||
class PluginTextEmbeddingNumTokensResponse(BaseModel):
|
||||
"""
|
||||
Response for number of tokens.
|
||||
"""
|
||||
|
||||
num_tokens: list[int] = Field(description="The number of tokens.")
|
||||
|
||||
|
||||
class PluginLLMNumTokensResponse(BaseModel):
|
||||
"""
|
||||
Response for number of tokens.
|
||||
"""
|
||||
|
||||
num_tokens: int = Field(description="The number of tokens.")
|
||||
|
||||
|
||||
class PluginStringResultResponse(BaseModel):
|
||||
result: str = Field(description="The result of the string.")
|
||||
|
||||
|
||||
class PluginVoiceEntity(BaseModel):
|
||||
name: str = Field(description="The name of the voice.")
|
||||
value: str = Field(description="The value of the voice.")
|
||||
|
||||
|
||||
class PluginVoicesResponse(BaseModel):
|
||||
voices: list[PluginVoiceEntity] = Field(description="The result of the voices.")
|
||||
|
||||
|
||||
class PluginDaemonError(BaseModel):
|
||||
"""
|
||||
Error from plugin daemon.
|
||||
"""
|
||||
|
||||
error_type: str
|
||||
message: str
|
||||
|
||||
|
||||
class PluginDaemonInnerError(Exception):
|
||||
code: int
|
||||
message: str
|
||||
|
||||
def __init__(self, code: int, message: str):
|
||||
self.code = code
|
||||
self.message = message
|
||||
|
||||
|
||||
class PluginInstallTaskStatus(StrEnum):
|
||||
Pending = "pending"
|
||||
Running = "running"
|
||||
Success = "success"
|
||||
Failed = "failed"
|
||||
|
||||
|
||||
class PluginInstallTaskPluginStatus(BaseModel):
|
||||
plugin_unique_identifier: str = Field(description="The plugin unique identifier of the install task.")
|
||||
plugin_id: str = Field(description="The plugin ID of the install task.")
|
||||
status: PluginInstallTaskStatus = Field(description="The status of the install task.")
|
||||
message: str = Field(description="The message of the install task.")
|
||||
icon: str = Field(description="The icon of the plugin.")
|
||||
labels: I18nObject = Field(description="The labels of the plugin.")
|
||||
|
||||
|
||||
class PluginInstallTask(BasePluginEntity):
|
||||
status: PluginInstallTaskStatus = Field(description="The status of the install task.")
|
||||
total_plugins: int = Field(description="The total number of plugins to be installed.")
|
||||
completed_plugins: int = Field(description="The number of plugins that have been installed.")
|
||||
plugins: list[PluginInstallTaskPluginStatus] = Field(description="The status of the plugins.")
|
||||
|
||||
|
||||
class PluginInstallTaskStartResponse(BaseModel):
|
||||
all_installed: bool = Field(description="Whether all plugins are installed.")
|
||||
task_id: str = Field(description="The ID of the install task.")
|
||||
|
||||
|
||||
class PluginUploadResponse(BaseModel):
|
||||
unique_identifier: str = Field(description="The unique identifier of the plugin.")
|
||||
manifest: PluginDeclaration
|
||||
206
api/core/plugin/entities/request.py
Normal file
206
api/core/plugin/entities/request.py
Normal file
@@ -0,0 +1,206 @@
|
||||
from typing import Any, Literal, Optional
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
from core.entities.provider_entities import BasicProviderConfig
|
||||
from core.model_runtime.entities.message_entities import (
|
||||
AssistantPromptMessage,
|
||||
PromptMessage,
|
||||
PromptMessageRole,
|
||||
PromptMessageTool,
|
||||
SystemPromptMessage,
|
||||
ToolPromptMessage,
|
||||
UserPromptMessage,
|
||||
)
|
||||
from core.model_runtime.entities.model_entities import ModelType
|
||||
from core.workflow.nodes.parameter_extractor.entities import (
|
||||
ModelConfig as ParameterExtractorModelConfig,
|
||||
)
|
||||
from core.workflow.nodes.parameter_extractor.entities import (
|
||||
ParameterConfig,
|
||||
)
|
||||
from core.workflow.nodes.question_classifier.entities import (
|
||||
ClassConfig,
|
||||
)
|
||||
from core.workflow.nodes.question_classifier.entities import (
|
||||
ModelConfig as QuestionClassifierModelConfig,
|
||||
)
|
||||
|
||||
|
||||
class RequestInvokeTool(BaseModel):
|
||||
"""
|
||||
Request to invoke a tool
|
||||
"""
|
||||
|
||||
tool_type: Literal["builtin", "workflow", "api"]
|
||||
provider: str
|
||||
tool: str
|
||||
tool_parameters: dict
|
||||
|
||||
|
||||
class BaseRequestInvokeModel(BaseModel):
|
||||
provider: str
|
||||
model: str
|
||||
model_type: ModelType
|
||||
|
||||
model_config = ConfigDict(protected_namespaces=())
|
||||
|
||||
|
||||
class RequestInvokeLLM(BaseRequestInvokeModel):
|
||||
"""
|
||||
Request to invoke LLM
|
||||
"""
|
||||
|
||||
model_type: ModelType = ModelType.LLM
|
||||
mode: str
|
||||
completion_params: dict[str, Any] = Field(default_factory=dict)
|
||||
prompt_messages: list[PromptMessage] = Field(default_factory=list)
|
||||
tools: Optional[list[PromptMessageTool]] = Field(default_factory=list)
|
||||
stop: Optional[list[str]] = Field(default_factory=list)
|
||||
stream: Optional[bool] = False
|
||||
|
||||
model_config = ConfigDict(protected_namespaces=())
|
||||
|
||||
@field_validator("prompt_messages", mode="before")
|
||||
@classmethod
|
||||
def convert_prompt_messages(cls, v):
|
||||
if not isinstance(v, list):
|
||||
raise ValueError("prompt_messages must be a list")
|
||||
|
||||
for i in range(len(v)):
|
||||
if v[i]["role"] == PromptMessageRole.USER.value:
|
||||
v[i] = UserPromptMessage(**v[i])
|
||||
elif v[i]["role"] == PromptMessageRole.ASSISTANT.value:
|
||||
v[i] = AssistantPromptMessage(**v[i])
|
||||
elif v[i]["role"] == PromptMessageRole.SYSTEM.value:
|
||||
v[i] = SystemPromptMessage(**v[i])
|
||||
elif v[i]["role"] == PromptMessageRole.TOOL.value:
|
||||
v[i] = ToolPromptMessage(**v[i])
|
||||
else:
|
||||
v[i] = PromptMessage(**v[i])
|
||||
|
||||
return v
|
||||
|
||||
|
||||
class RequestInvokeTextEmbedding(BaseRequestInvokeModel):
|
||||
"""
|
||||
Request to invoke text embedding
|
||||
"""
|
||||
|
||||
model_type: ModelType = ModelType.TEXT_EMBEDDING
|
||||
texts: list[str]
|
||||
|
||||
|
||||
class RequestInvokeRerank(BaseRequestInvokeModel):
|
||||
"""
|
||||
Request to invoke rerank
|
||||
"""
|
||||
|
||||
model_type: ModelType = ModelType.RERANK
|
||||
query: str
|
||||
docs: list[str]
|
||||
score_threshold: float
|
||||
top_n: int
|
||||
|
||||
|
||||
class RequestInvokeTTS(BaseRequestInvokeModel):
|
||||
"""
|
||||
Request to invoke TTS
|
||||
"""
|
||||
|
||||
model_type: ModelType = ModelType.TTS
|
||||
content_text: str
|
||||
voice: str
|
||||
|
||||
|
||||
class RequestInvokeSpeech2Text(BaseRequestInvokeModel):
|
||||
"""
|
||||
Request to invoke speech2text
|
||||
"""
|
||||
|
||||
model_type: ModelType = ModelType.SPEECH2TEXT
|
||||
file: bytes
|
||||
|
||||
@field_validator("file", mode="before")
|
||||
@classmethod
|
||||
def convert_file(cls, v):
|
||||
# hex string to bytes
|
||||
if isinstance(v, str):
|
||||
return bytes.fromhex(v)
|
||||
else:
|
||||
raise ValueError("file must be a hex string")
|
||||
|
||||
|
||||
class RequestInvokeModeration(BaseRequestInvokeModel):
|
||||
"""
|
||||
Request to invoke moderation
|
||||
"""
|
||||
|
||||
model_type: ModelType = ModelType.MODERATION
|
||||
text: str
|
||||
|
||||
|
||||
class RequestInvokeParameterExtractorNode(BaseModel):
|
||||
"""
|
||||
Request to invoke parameter extractor node
|
||||
"""
|
||||
|
||||
parameters: list[ParameterConfig]
|
||||
model: ParameterExtractorModelConfig
|
||||
instruction: str
|
||||
query: str
|
||||
|
||||
|
||||
class RequestInvokeQuestionClassifierNode(BaseModel):
|
||||
"""
|
||||
Request to invoke question classifier node
|
||||
"""
|
||||
|
||||
query: str
|
||||
model: QuestionClassifierModelConfig
|
||||
classes: list[ClassConfig]
|
||||
instruction: str
|
||||
|
||||
|
||||
class RequestInvokeApp(BaseModel):
|
||||
"""
|
||||
Request to invoke app
|
||||
"""
|
||||
|
||||
app_id: str
|
||||
inputs: dict[str, Any]
|
||||
query: Optional[str] = None
|
||||
response_mode: Literal["blocking", "streaming"]
|
||||
conversation_id: Optional[str] = None
|
||||
user: Optional[str] = None
|
||||
files: list[dict] = Field(default_factory=list)
|
||||
|
||||
|
||||
class RequestInvokeEncrypt(BaseModel):
|
||||
"""
|
||||
Request to encryption
|
||||
"""
|
||||
|
||||
opt: Literal["encrypt", "decrypt", "clear"]
|
||||
namespace: Literal["endpoint"]
|
||||
identity: str
|
||||
data: dict = Field(default_factory=dict)
|
||||
config: list[BasicProviderConfig] = Field(default_factory=list)
|
||||
|
||||
|
||||
class RequestInvokeSummary(BaseModel):
|
||||
"""
|
||||
Request to summary
|
||||
"""
|
||||
|
||||
text: str
|
||||
instruction: str
|
||||
|
||||
|
||||
class RequestRequestUploadFile(BaseModel):
|
||||
"""
|
||||
Request to upload file
|
||||
"""
|
||||
|
||||
filename: str
|
||||
mimetype: str
|
||||
Reference in New Issue
Block a user