Developing Inventory Plugin

This document guides you through developing a plugin for Inventory, one of SpaceONE’s microservices. This example demonstrates how to collect information about the top 5 cryptocurrencies by price using the Portfolio plugin, which manages and tracks your assets.

1. Environment Setup

❗️

Before proceeding with plugin development, make sure to complete SpaceONE Installation first!

Specific environment configurations are required before plugin development. Please refer to the table below for required libraries and supported versions.

Name	Version Info
spaceone-inventory	2.0.dev210

2. Environment Setup

1) Package Installation

sudo pip3 install spaceone-inventory --pre --upgrade

2) Directory Creation

mkdir plugin-portfolio-inven-collector && cd $_

3) Inventory Plugin Project Creation

spaceone create-project src/plugin -s asset

src/plugin directory will contain the following files:

- - __init__.py
  - main.py

3. main.py File Configuration

1) `collector_init` Function

ℹ️

collector_init function is a function to initialize the plugin. This function is called when the plugin starts.

For the plugin-portfolio-inven-collector example, collector_init function does not require options_schema, so it is written as follows.

src/plugin/main.py

from spaceone.inventory.plugin.collector.lib.server import CollectorPluginServer

app = CollectorPluginServer()


@app.route("Collector.init")
def collector_init(params: dict) -> dict:
    """init plugin by options

    Args:
        params (CollectorInitRequest): {
            'options': 'dict',    # Required
            'domain_id': 'str'
        }

    Returns:
        PluginResponse: {
            'metadata': 'dict'
        }
    """
    return {"metadata": {"options_schema": {}}}

2) `collector_collect` Function

ℹ️

Before writing collector_collect function, let’s create a manager package to handle business logic.

mkdir -p src/plugin/manager touch src/plugin/manager/__init__.py touch src/plugin/manager/cryptocurrency_manager.py

2-1) The directory/file structure created by the above commands is as follows:

- - - __init__.py
    - cryptocurrency_manager.py
  - __init__.py
  - main.py

2-2) First, please write the skeleton code as follows:

src/plugin/manager/cryptocurrency_manager.py

import logging

from spaceone.core.manager import BaseManager

_LOGGER = logging.getLogger(__name__)


class CryptoManager(BaseManager):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    def collect_resources(self, options, secret_data, schema):
        pass

2-3) Back to main.py to write collector_collect function:

src/plugin/main.py

from typing import Generator

from spaceone.inventory.plugin.collector.lib.server import CollectorPluginServer

from plugin.manager.crypto_manager import CryptoManager

...

@app.route("Collector.collect")
def collector_collect(params: dict) -> Generator[dict, None, None]:
    ...
    
    options = params["options"]
    secret_data = params["secret_data"]
    schema = params.get("schema")

    crypto_mgr = CryptoManager()
    return crypto_mgr.collect_resources(options, secret_data, schema)

4. Business Logic Writing and External Connection

Now let’s write business logic to collect cryptocurrency information:

1) Manager

1-1) `init`

src/plugin/manager/cryptocurrency_manager.py

import logging
import os

from spaceone.core.manager import BaseManager

_LOGGER = logging.getLogger(__name__)
_CURRENT_DIR = os.path.dirname(__file__)
_METADATA_DIR = os.path.join(_CURRENT_DIR, "../metadata/")


class CryptoManager(BaseManager):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self.provider = "portfolio"
        self.cloud_service_group = "Investment"
        self.cloud_service_type = "Cryptocurrency"
        self.metadata_path = os.path.join(
            _METADATA_DIR, "investment/cryptocurrency.yaml"
        )
    ...

provider > cloud_service_group > cloud_service_type

provider means cloud service providers like AWS, Azure, Google Cloud.
- For simplicity, we set it to ‘portfolio’.

cloud_service_group is a sub-concept of provider located at the top of the left sidebar.
cloud_service_type is a sub-concept of cloud service group, and there may be multiple types under a cloud service group.
metadata_path means the path to store cloud service type metadata.
- This metadata can be used to make cloud service type more rich.

1-2) `collect_resources`

Please write the skeleton code as follows:

src/plugin/manager/cryptocurrency_manager.py

import logging

from spaceone.core.manager import BaseManager
from spaceone.inventory.plugin.collector.lib import (
    make_cloud_service_type,
    make_error_response,
    make_response,
)

_LOGGER = logging.getLogger(__name__)


class CryptoManager(BaseManager):
    ...
     
    def collect_resources(self, options, secret_data, schema):
        try:
            yield from self.collect_cloud_service_type(options, secret_data, schema)
            yield from self.collect_cloud_service(options, secret_data, schema)
        except Exception as e:
            yield make_error_response(
                error=e,
                provider=self.provider,
                cloud_service_group=self.cloud_service_group,
                cloud_service_type=self.cloud_service_type,
            )

    def collect_cloud_service_type(self, options, secret_data, schema):
        cloud_service_type = make_cloud_service_type(
            name=self.cloud_service_type,
            group=self.cloud_service_group,
            provider=self.provider,
            metadata_path=self.metadata_path,
            is_primary=True,
            is_major=True,
        )

        yield make_response(
            cloud_service_type=cloud_service_type,
            match_keys=[["name", "reference.resource_id", "account", "provider"]],
            resource_type="inventory.CloudServiceType",
        )

    def collect_cloud_service(self, options, secret_data, schema):
        pass

collect_cloud_service method needs to be written before collect_cloud_service method in cryptocurrency_manager.py.

2) Connector

2-1) Create connector package:

mkdir -p src/plugin/connector touch src/plugin/connector/__init__.py touch src/plugin/connector/cryptocurrency_connector.py

2-2) The directory/file structure created by the above commands is as follows:

- - - __init__.py
    - cryptocurrency_connector.py
  - - __init__.py
    - cryptocurrency_manager.py
  - __init__.py
  - main.py

2-3) Now let’s use pycoingecko library to get information about the top 5 cryptocurrencies by price at the current time (coins):

pip3 install pycoingecko

src/plugin/connector/cryptocurrency_connector.py

import logging
from typing import Dict, List

from pycoingecko import CoinGeckoAPI
from spaceone.core.connector import BaseConnector

_LOGGER = logging.getLogger(__name__)


class CryptoConnector(BaseConnector):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.client = CoinGeckoAPI()

    def list_cryptocurrencies(self) -> List[Dict]:
        try:
            coins = self.client.get_coins_markets(
                vs_currency="krw", order="market_cap_desc", per_page=5, page=1
            )

            filtered_coins = list()
            for coin in coins:
                filtered_coins.append(
                    {
                        "name": coin["name"],
                        "current_price": coin["current_price"],
                        "market_cap_rank": coin["market_cap_rank"],
                        "price_change_percentage_24h": coin[
                            "price_change_percentage_24h"
                        ],
                        "high_24h": coin["high_24h"],
                        "low_24h": coin["low_24h"],
                        "last_updated": coin["last_updated"],
                    }
                )
            return filtered_coins
        except Exception as e:
            _LOGGER.error(f"Error fetching cryptocurrency data: {e}")
            return []

🧐

In the above code, filtered_coins contains only the information that will be shown to the user from the Raw Data received from the get_coins_markets method of the pycoingecko library.

For reference, the Raw Data information that goes into coins is as follows.

[
  {
    "id": "bitcoin",
    "symbol": "btc",
    "name": "Bitcoin",
    "image": "https://coin-images.coingecko.com/coins/images/1/large/bitcoin.png?1696501400",
    "current_price": 89860690,
    "market_cap": 1773045681619077,
    "market_cap_rank": 1,
    "fully_diluted_valuation": 1887089127326028,
    "total_volume": 53990231060500,
    "high_24h": 92076398,
    "low_24h": 87861025,
    "price_change_24h": -1924582.0696388185,
    "price_change_percentage_24h": -2.09683,
    "market_cap_change_24h": -36837579495029.25,
    "market_cap_change_percentage_24h": -2.03536,
    "circulating_supply": 19730896.0,
    "total_supply": 21000000.0,
    "max_supply": 21000000.0,
    "ath": 98576718,
    "ath_change_percentage": -8.99866,
    "ath_date": "2024-06-07T13:55:20.414Z",
    "atl": 75594,
    "atl_change_percentage": 118568.64714,
    "atl_date": "2013-07-05T00:00:00.000Z",
    "roi": None,
    "last_updated": "2024-07-25T17:07:53.002Z",
  }

  ...
]

2-4) Back to cryptocurrency_connector’s call part in cryptocurrency_manager.py to complete the code:

3) Manager

src/plugin/manager/cryptocurrency_manager.py

...

from spaceone.inventory.plugin.collector.lib import (
    make_cloud_service_type,
    make_cloud_service_with_metadata,
    make_error_response,
    make_response,
)

from plugin.connector.cryptocurrency_connector import CryptoConnector

...

class CryptoManager(BaseManager):
    ...
    
    def collect_cloud_service(self, options, secret_data, schema):
        crypto_connector = CryptoConnector()
        cryptocurrencies = crypto_connector.list_cryptocurrencies()

        for crypto in cryptocurrencies:
            cloud_service = make_cloud_service_with_metadata(
                name=crypto["name"],
                cloud_service_type=self.cloud_service_type,
                cloud_service_group=self.cloud_service_group,
                provider=self.provider,
                data=crypto,
                data_format="dict",
                metadata_path=self.metadata_path,
            )
            yield make_response(
                cloud_service=cloud_service,
                match_keys=[["name", "reference.resource_id", "account", "provider"]],
            )

5. Metadata YAML File Writing

Please write a yaml file to provide filtered information to users:

mkdir -p src/plugin/metadata/investment touch src/plugin/metadata/investment/cryptocurrency.yaml

⚠️

The path of the yaml file must follow the src/plugin/metadata/[cloud_service_group]/[cloud_service_type].yaml convention. Therefore, you must follow this convention when developing.

In the following yaml file, you can see that all filtered data information goes into the data. format as value:

src/plugin/metadata/investment/cryptocurrency.yaml

search:
  fields:
    - Name: data.name
    - Market Cap Rank: data.market_cap_rank
table:
  sort:
    key: data.market_cap_rank
    desc: true
  fields:
    - Name: data.name
    - Price: data.current_price
    - Last Updated (UTC): data.last_updated
    - Market Cap Rank: data.market_cap_rank
    - 24h Change (%): data.price_change_percentage_24h
    - 24h Low: data.low_24h
    - 24h High: data.high_24h
tabs.0:
  name: Details
  type: item
  fields:
    - Name: data.name
    - Price: data.current_price
    - Last Updated (UTC): data.last_updated
    - Market Cap Rank: data.market_cap_rank
    - 24h Change (%): data.price_change_percentage_24h
    - 24h Low: data.low_24h
    - 24h High: data.high_24h

6. Execution

The plugin development preparation is complete. Now let’s perform a test using spacectl command and a simple YAML file:

1) Plugin (gRPC) Server Execution

You can execute the server using PyCharm or CLI:

1-1) PyCharm Usage

First, set src directory as ‘Sources Root’:

Run/Debug Configurations, set it as follows, and use ‘Run’ button to execute the plugin server:

1-2) CLI Usage

The result of the plugin server execution is as follows:

spaceone run plugin-server -s src plugin

2024-07-16T10:48:44.515Z [DEBUG]       (server.py:69) Loaded Services: 
         - spaceone.api.inventory.plugin.Collector
         - spaceone.api.inventory.plugin.Job
         - grpc.health.v1.Health
         - spaceone.api.core.v1.ServerInfo  
2024-07-16T10:48:44.515Z [INFO]       (server.py:73) Start gRPC Server (plugin): port=50051, max_workers=100

2) `init` Test

init.yaml

---
options: { }

2-1) Use a new CLI tab to enter the following command while the plugin server is running:

spacectl exec init plugin.Collector -f init.yaml

The result of init is as follows.

---
metadata:
  filter_format: [ ]
  options_schema: { }
  supported_features:
    - garbage_collection
  supported_resource_type:
    - inventory.CloudService
    - inventory.CloudServiceType
    - inventory.Region
    - inventory.Namespace
    - inventory.Metric
  supported_schedules:
    - hours

3) `collect` Test

collect.yaml

---
options: { }
secret_data: { }

spacectl exec collect plugin.Collector -f collect.yaml

3-1) The result of the above command is as follows:

Cloud Service Type: Cryptocurrency
5 instances of Cryptocurrency

The result of collect is as follows.

---
match_rules:
  '1':
  - name
  - reference.resource_id
  - account
  - provider
resource:
  group: Investment
  is_major: true
  is_primary: true
  json_metadata: '{"view": {"search": [{"name": "Name", "key": "data.name", "type":
    "text"}, {"name": "Market Cap Rank", "key": "data.market_cap_rank", "type": "text"}],
    "table": {"layout": {"type": "query-search-table", "options": {"default_sort":
    {"key": "data.market_cap_rank", "desc": true}, "fields": [{"name": "Name", "key":
    "data.name", "type": "text"}, {"name": "Price", "key": "data.current_price", "type":
    "text"}, {"name": "Last Updated (UTC)", "key": "data.last_updated", "type": "text"},
    {"name": "Market Cap Rank", "key": "data.market_cap_rank", "type": "text"}, {"name": "24h
    Change (%)", "key": "data.price_change_percentage_24h", "type": "text"}, {"name":
    "24h Low", "key": "data.low_24h", "type": "text"}, {"name": "24h High", "key":
    "data.high_24h", "type": "text"}]}, "name": "Main Table"}}, "sub_data": {"layouts":
    [{"name": "Details", "type": "item", "options": {"fields": [{"name": "Name", "key":
    "data.name", "type": "text"}, {"name": "Price", "key": "data.current_price", "type":
    "text"}, {"name": "Last Updated (UTC)", "key": "data.last_updated", "type": "text"},
    {"name": "Market Cap Rank", "key": "data.market_cap_rank", "type": "text"}, {"name":
    "24h Change (%)", "key": "data.price_change_percentage_24h", "type": "text"}, {"name":
    "24h Low", "key": "data.low_24h", "type": "text"}, {"name": "24h High", "key":
    "data.high_24h", "type": "text"}]}}]}}}'
  labels: []
  metadata: null
  name: Cryptocurrency
  provider: portfolio
  service_code: null
  tags: {}
resource_type: inventory.CloudServiceType
state: SUCCESS

---
match_rules:
  '1':
  - name
  - reference.resource_id
  - account
  - provider
resource:
  account: null
  cloud_service_group: Investment
  cloud_service_type: Cryptocurrency
  data:
    current_price: 89860690.0
    high_24h: 92076398.0
    last_updated: '2024-07-25T17:07:53.002Z'
    low_24h: 87861025.0
    market_cap_rank: 1.0
    name: Bitcoin
    price_change_percentage_24h: -2.09683
  instance_size: 0.0
  instance_type: null
  ip_addresses: []
  json_data: null
  json_metadata: null
  launched_at: null
  metadata:
    view:
      search:
      - key: data.name
        name: Name
        type: text
      - key: data.market_cap_rank
        name: Market Cap Rank
        type: text
      sub_data:
        layouts:
        - name: Details
          options:
            fields:
            - key: data.name
              name: Name
              type: text
            - key: data.current_price
              name: Price
              type: text
            - key: data.last_updated
              name: Last Updated (UTC)
              type: text
            - key: data.market_cap_rank
              name: Market Cap Rank
              type: text
            - key: data.price_change_percentage_24h
              name: 24h Change (%)
              type: text
            - key: data.low_24h
              name: 24h Low
              type: text
            - key: data.high_24h
              name: 24h High
              type: text
          type: item
      table:
        layout:
          name: Main Table
          options:
            default_sort:
              desc: true
              key: data.market_cap_rank
            fields:
            - key: data.name
              name: Name
              type: text
            - key: data.current_price
              name: Price
              type: text
            - key: data.last_updated
              name: Last Updated (UTC)
              type: text
            - key: data.market_cap_rank
              name: Market Cap Rank
              type: text
            - key: data.price_change_percentage_24h
              name: 24h Change (%)
              type: text
            - key: data.low_24h
              name: 24h Low
              type: text
            - key: data.high_24h
              name: 24h High
              type: text
          type: query-search-table
  name: Bitcoin
  provider: portfolio
  reference: null
  region_code: null
  tags: {}
resource_type: inventory.CloudService
state: SUCCESS

...

3-2) If you check the information of one of the instances of Cryptocurrency, such as Bitcoin, in the resource field, you can see that filtered information goes into the data field:

And in the metadata field, you can see the filtered information in the Detail of each instance:

This completes the SpaceONE Cryptocurrency plugin development. You can start plugin development by referring to the above example.

If you want to show the developed plugin to users through the UI, please refer to the next page’s plugin registration method:

Registering Inventory Plugin

Developing Inventory Plugin

1. Environment Setup

2. Environment Setup

1) Package Installation

2) Directory Creation

3) Inventory Plugin Project Creation

3. main.py File Configuration

1) collector_init Function

2) collector_collect Function

4. Business Logic Writing and External Connection

1) Manager

1-1) __init__

1-2) collect_resources

2) Connector

3) Manager

5. Metadata YAML File Writing

6. Execution

1) Plugin (gRPC) Server Execution

2) init Test

3) collect Test

1) `collector_init` Function

2) `collector_collect` Function

1-1) `init`

1-2) `collect_resources`

2) `init` Test

3) `collect` Test