1. Patch, Minor, or Major is selected during workflow dispatch (defaults
to patch)
2. Get most recent release version
3. Calculate new version based on findings from step 2
4. Build wheels
5. Build worker and base-worker
6. Push to GHCR
7. Create new release with release notes
Also, I removed everything related to ECR
This is the first of a few PRs. Deploy to staging will fail until we
have `arcade-core`, `arcade-serve`, and `arcade-ai` released to PyPI.
This PR will release `arcade-core` to PyPI.
### PR Description
* Adds workflow that checks for changes in any pyproject.toml, and if
its version has changed, then tests, builds wheel, then publishes to
PyPI
* Updates the Dockerfile for our new structure
* Updates porter yamls
* Updates `make full-dist`
* Removes a couple unused workflows
Check out https://github.com/ArcadeAI/arcade-ai/actions/runs/15622059209
to see how the new workflow works (note that it failed publishing to
PyPI on purpose)
12 tools for the new Confluence toolkit.
| Name | Description |
|----------------------------------|------------------------------------------------------------------------------------|
| Confluence.CreatePage | Create a new page at the root of the given
space. |
| Confluence.UpdatePageContent | Update a page's content. |
| Confluence.RenamePage | Rename a page by changing its title. |
| Confluence.GetPage | Retrieve a SINGLE page's content by its ID or
title. |
| Confluence.GetPagesById | Get the content of MULTIPLE pages by their
ID in a single efficient request. |
| Confluence.ListPages | Get the content of multiple pages by their ID.
|
| Confluence.ListAttachments | List attachments in a workspace. |
| Confluence.GetAttachmentsForPage | Get attachments for a page by its
ID or title. |
| Confluence.SearchContent | Search for content in Confluence. |
| Confluence.GetSpace | Get the details of a space by its ID or key. |
| Confluence.ListSpaces | List all spaces sorted by name in ascending
order. |
| Confluence.GetSpaceHierarchy | Retrieve the full hierarchical
structure of a Confluence space as a tree structure.|
### Confluence Clients
Confluence has deprecated most of their V1 endpoints, so most of the
tools use V2. However, we still need a V1 client to support search. The
V1 search API has not been deprecated yet, because there is no V2
equivalent. But we need to be aware in the future for when this
deprecation happens.
### Future work
* Content of pages are returned in the Confluence `storage` format. This
is the format that Confluence uses to store pages internally. We should
understand the storage format more deeply, and write utility function to
transform this format into plain text.
* Better protections against extremely large pages. I've tested up to
6,000 word pages.
* Tools for blog posts
* Tool for getting the children of a page. (`get_space_hierarchy` will
suffice for now)
* Allow for numerical titles
Add hot-reload capability
adds watchfiles dependency and changes uvicorn server logic
but does not have any breaking changes.
---------
Co-authored-by: Eric Gustin <eric@arcade.dev>
Co-authored-by: sdreyer <sterling@arcade-ai.com>
Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
The Asana toolkit is not published yet, so it cannot be added to the
worker's toolkit list. This is causing container build/publish on main
branch to fail.
Reviewer:
1. DM me to get a client ID/secret for the Hubspot app and login
credentials for a sample Hubspot account
2. For now, you'll need to run the engine on the branch
`nate/token-introspection` for the auth flow to work with Hubspot
3. Add the following to `auth.providers` in your engine yaml:
```yaml
- id: arcade-hubspot
description: 'Hubspot provider'
enabled: true
type: oauth2
provider_id: hubspot
client_id: ${env:HUBSPOT_CLIENT_ID}
client_secret: ${env:HUBSPOT_CLIENT_SECRET}
```
## PR Description
Changes `pip install 'arcade-ai[fastapi]'` to `pip install arcade-ai`.
In other words, FastAPI is now a required dependecy of arcade-ai.
Additionally, I snuck in some minor cleanup changes.
This PR enhances the Docker build and deployment process for the Arcade
Worker by:
- **Modularizing Docker Builds:**
- Introduces a new `INSTALL_TOOLKITS` build argument in the `Dockerfile`
to conditionally include toolkits. this enables the creation of a
`arcadeai/worker-base` which can be used to build custom containers in a
multi-stage build. an example of this is included in the example dir.
- Adds `docker-base` Makefile target to build a lightweight base image
without toolkits.
- **Publishing to GitHub Container Registry (GHCR):**
- Adds Makefile targets `publish-ghcr` and `gh-login` for pushing images
to GHCR.
- Supports publishing both base and full images with toolkits to GHCR.
- **Docker Compose:**
- Add Docker compose file and setup
- Renames the `actor` service to `worker`
- Adds an `nginx` service in `docker-compose.yml` to proxy requests to
the Arcade Engine.
- Introduces an `nginx.conf` file for the Nginx service.
- **Streamlining Toolkit Installation:**
- Moves toolkit installation from the `start.sh` script to the Docker
build process.
- Creates a `toolkits.txt` file to manage toolkit dependencies which can
be edited easily when we want to add a new toolkit. Developers can also
use this approach as shown in the example.
- **Improving Configuration Files:**
- Updates `docker.engine.yaml` and `env.example` to align with the new
setup.
TODO:
- CI/CD needs to be adjusted so that images are pushed to ghcr on
release.
- AWS resources need to be renamed actor -> worker @EricGustin
This can go in after the above two items are resolved.
---------
Co-authored-by: Wils Dawson <wils@arcade-ai.com>
Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
Co-authored-by: sdreyer <sterling@arcade-ai.com>
Co-authored-by: Sterling Dreyer <sdreyer21@gmail.com>
Co-authored-by: Nate Barbettini <nathanaelb@gmail.com>
This PR improves the Docker build process by shifting from building the
project within the Docker image to using pre-built wheels. The main
changes are:
1. **Updated Makefile:**
- **`VERSION` Variable:** Set to `0.1.0.dev0` to reflect the new default
development version.
- **`docker` Target:**
- Added steps to build the Arcade and toolkit wheels before building the
Docker image.
- Exports the required extras (`fastapi`, `evals`) to a
`requirements.txt` file.
- **`full-dist` Target:**
- Builds distributions for the main project and all toolkits.
- Copies all the built wheels to a centralized `./dist` directory.
- **`clean-dist` Target:**
- Cleans build artifacts from `./dist`, `arcade/dist`, and
`toolkits/*/dist` directories.
2. **Modified Dockerfile:**
- **Copy Pre-built Wheels:** Adjusted to copy wheels and the
`requirements.txt` from the `./dist` directory into the Docker image.
- **Installation Process:**
- Installs the Arcade wheel with the necessary extras.
- Installs toolkits from the copied wheel files, eliminating the need to
build them inside the Docker image.
- **Simplification:** Removed unnecessary commands, such as installing
build tools and copying the entire codebase, to streamline the
Dockerfile.
3. **Toolkits `pyproject.toml` Updates:**
- Changed the `arcade-ai` dependency version from `^0.1.0` to `0.1.*` in
all toolkit `pyproject.toml` files to ensure compatibility with the new
versioning scheme.
4. **Docker Makefile Adjustments:**
- Set the `VERSION` variable to `0.1.0.dev0` to align with the main
Makefile.
- Ensures consistent versioning across Docker-related build processes.
**Benefits:**
- **Efficiency:** Building wheels outside the Docker context reduces the
Docker image build time and resource consumption. overall docker image
size reduced by **1Gb**!!!
- **Reliability:** Using pre-built wheels ensures consistency across
different environments and simplifies dependency management.
- **Maintainability:** The Dockerfile and Makefiles are cleaner and more
straightforward, making them easier to understand and maintain.
**Notes:**
- Developers should run `make docker` to build and run the Docker
container using the new process.
- Ensure that any CI/CD pipelines are updated to accommodate these
changes in the build process. @sdreyer
This PR includes several improvements to the Arcade client and adds
LangGraph examples:
1. Enhanced error handling in the Arcade client:
- Improved HTTP error handling in `BaseArcadeClient`
- Simplified request methods in `SyncArcadeClient` and
`AsyncArcadeClient`
2. Updated `ToolResource` class:
- Changed base path from `/v1/tool` to `/v1/tools`
- Added `tool_version` parameter to `authorize` method
3. Improved Toolkit discovery:
- Updated `find_all_arcade_toolkits` to search only in the current
Python interpreter's site-packages
5. Added LangGraph examples:
- New `langgraph_auth.py` example demonstrating Gmail authentication
- New `langgraph_with_tool_exec.py` example showing tool execution
within a LangGraph
6. Minor updates:
- Changed default `BASE_URL` to `https://api.arcade.com/`
- Updated import error message for eval dependencies
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
* Renamed `arcade_arithmetic` to `arcade_math`
* Deleted `arcade_github` toolkit for the next release. This will be
reintroduced later.
* Added 5 tools to `arcade_x` toolkit
- post_tweet
- delete_tweet_by_id
- search_recent_tweets_by_username
- search_recent_tweets_by_keywords
- lookup_single_user_by_username
# PR Description
## Summary
Changes include renaming the `arcade_gmail` toolkit to `arcade_google`,
adding unit tests for Google toolkit, add new tools to the Google
toolkit.
## Changes
### Makefile
- Added a new `make test-toolkits` target to iterate over all toolkits
and run pytest on each one.
### Added new tools for the google toolkit
1. `send_email`
This tool sends an email using the Gmail API.
2. `write_draft_email`
This tool creates a draft email using the Gmail API.
3. `update_draft_email`
This tool updates an existing draft email using the Gmail API.
4. `send_draft_email`
This tool sends a draft email using the Gmail API.
5. `delete_draft_email`
This tool deletes a draft email using the Gmail API.
6. `list_draft_emails`
This tool retrieves a list of draft emails using the Gmail API.
7. `list_emails_by_header`
This tool searches for emails by a specific header using the Gmail API.
- `sender`: The sender's email address to search for.
- `limit`: The maximum number of emails to retrieve.
8. `list_emails`
This tool retrieves a list of emails using the Gmail API.
9. `trash_email`
This tool moves an email to the trash using the Gmail API.
#### Summary
This PR introduces key updates to the Arcade AI codebase, focusing on
improving the CLI tool, refining the Docker build process, and enhancing
documentation within the Gmail toolkit.
#### Key Changes:
1. **Docker Build**:
- Added the `--no-cache` option to the `docker-build` target in the
Makefile to ensure fresh builds, preventing issues related to cached
layers.
2. **CLI Tool**:
- Introduced a new optional `prompt` parameter in the `chat` command,
allowing users to customize the system role's prompt. A default prompt
is now provided if none is specified.
3. **Gmail Toolkit**:
- Updated the `write_draft` function with improved docstrings for
clearer guidance and maintainability.
#### Impact:
- **Developer Workflow**: Improved Docker reliability and enhanced CLI
flexibility.
- **User Experience**: More customizable interactions in the CLI and
better documentation in the Gmail toolkit.
Please review and provide feedback.
