Today, tweaking your PC to suit your workflows often involves digging through menus and settings across multiple control panels. Project G-Assist is an experimental on-device AI assistant designed to help users control their RTX GPU and other PC settings using a natural language interface. G-Assist can execute common NVIDIA App commands, optimize performance, monitor performance statistics, and much more. Plus, G-Assist can be used while you’re gaming or creating thanks to the NVIDIA App Overlay, without having to alt-tab. 

Most importantly, G-Assist can be extended by the community via plug-ins to control other hardware or apps in your PC, such as your mouse settings, what’s playing on Spotify, or even your smart devices with IFTTT. 

G-Assist provides an accessible way to build AI-powered routines on your PC. It provides the infrastructure, including a small language model (SLM) that can run locally, and the developer just has to provide the functions it wants to trigger. This is made even simpler thanks to a GPT-based Plug-in Builder. 

In this blog post, we’ll show you how you can create your own G-Assist plug-in using a Twitch integration as an example. You’ll learn how plug-ins work, how they communicate with G-Assist, and how to build your own from scratch. Whether you’re a Python dev, C++ enthusiast, or just getting started, we’ve got tools and templates to make plug-in development fast and approachable.

What are G-Assist plug-ins?

ninja is LIVE!
Title: Friday Fortnite!
Game: Fortnite
Viewers: 45,231
Started At: 2024-03-14T12:34:56Z

{
  "manifestVersion": 1,
  "executable": "g-assist-plug-in-twitch.exe",
  "persistent": false,
  "functions": [
    {
      "name": "check_twitch_live_status",
      "description": "Checks if a Twitch user is live and retrieves stream details.",
      "tags": ["twitch", "live_status"],
      "properties": {
        "username": {
          "type": "string",
          "description": "The Twitch username to check."
        }
      }
    }
  ]
}

{
  "TWITCH_CLIENT_ID": "your_client_id_here",
  "TWITCH_CLIENT_SECRET": "your_client_secret_here"
}

def main():
    setup_logging()
    logging.info("Twitch Plugin Started")

    while True:
        command = read_command()
        if command is None:
            continue

        for tool_call in command.get("tool_calls", []):
            func = tool_call.get("func")
            params = tool_call.get("params", {})
            if func == "check_twitch_live_status":
                response = check_twitch_live_status(params)
                write_response(response)
            elif func == "shutdown":
                return

{
  "tool_calls": [{
    "func": "check_twitch_live_status",
    "params": {
      "username": "nvidia"
    }
  }]
}

{
  "success": true,
  "message": "nvidia is LIVE!\nTitle: NVIDIA Gaming Stream\nGame: Cyberpunk 2077\nViewers: 1234"
}>

def get_oauth_token():
    try:
        response = requests.post(
            TWITCH_OAUTH_URL,
            params={
                "client_id": config.get("TWITCH_CLIENT_ID"),
                "client_secret": config.get("TWITCH_CLIENT_SECRET"),
                "grant_type": "client_credentials"
            }
        )
        return response.json().get("access_token")
    except Exception as e:
        logging.error(f"Error getting OAuth token: {e}")
        return None

def check_twitch_live_status(params: Dict[str, str]):
    username = params.get("username")
    oauth_token = get_oauth_token()

    headers = {
        "Client-ID": config.get("TWITCH_CLIENT_ID"),
        "Authorization": f"Bearer {oauth_token}"
    }

    response = requests.get(
        TWITCH_STREAM_URL,
        headers=headers,
        params={"user_login": username}
    )

    data = response.json().get("data", [])
    if data:
        stream = data[0]
        return {
            "success": True,
            "message": (
                f"{username} is LIVE!\n"
                f"Title: {stream['title']}\n"
                f"Game: {stream.get('game_name', 'Unknown')}\n"
                f"Viewers: {stream['viewer_count']}"
            )
        }
    return {"success": True, "message": f"{username} is OFFLINE"}