Machine Learning in Production

The average #MachineLearning Engineer spends 98% of their time stitching services together.
*@abacusai can take data from you, train a model, deploy it and give you an endpoint you can use right away.
*

 

Less complexity = More productive #DataScience.
————#AI #DataScientists #ML pic.twitter.com/w3rbut608r

— Kirk Borne (@KirkDBorne) November 27, 2024

APIs Testing

I want a way of being able to post a YouTube video onto Blogger, Facebook, X, and Tumblr. In order to do it I needed to get API's for each service.  These APIs will make it possible for my scripts to talk to these platforms.

Blogger API

1. Create a Google Account: If you don't have one, you'll need to create a Google Account.

2. Visit the Google Developers Console: Go to the and sign in with your Google Account.

3. Create a Project: Click on "Create Project" and give it a name.

4. Enable the Blogger API: In the API Library, search for "Blogger" and enable the API for your project.

5. Create Credentials: Go to the Credentials page, click on "Create credentials," and select "API key."

6. Set Up OAuth 2.0: Follow the instructions to set up OAuth 2.0 for authorization.

Facebook API

1. Create a Facebook Developer Account: Go to the and log in with your Facebook account.

2. Create a New App: Click on "My Apps" and then "Create App."

3. Set Up the API: Choose the Facebook API and follow the setup instructions.

4. Generate Access Tokens: Go to the "Tools" section and use the Access Token Tool to generate user access tokens.

X (formerly Twitter) API

1. Create an X Account: Go to and sign up with your email or phone number.

2. Visit the Developer Portal: Go to the and log in with your X account.

3. Create an App: Click on "Create an App" and follow the instructions to set up your app.

4. Generate API Keys: Go to the "Keys and Tokens" tab to generate your API keys.

Tumblr API

1. Create a Tumblr Account: If you don't have one, sign up for a Tumblr account.

2. Visit the Tumblr Developer Portal: Go to the and log in with your Tumblr account.

3. Create an App: Click on "Create an App" and follow the instructions to set up your app.

4. Generate API Keys: Go to the "API Keys" section to generate your API keys.

OpenAI API

1. Visit the OpenAI Platform: Go to the .

2. Sign Up: Click on the "Sign Up" button and fill in the registration form with your basic information.

3. Verify Your Email: OpenAI will send a verification email. Verify your email address to complete the sign-up process.

4. Create API Keys: After signing up, go to your profile icon and select "View API Keys." Click "Create New Secret Key" to generate a new API key.

Gemini API

1. Visit the Gemini API Page: Go to the .

2. Sign In: Sign in with your Google account.

3. Create an API Key: In Google AI Studio, go to the API section and create a new API key.

4. Set Up Your Environment: Configure your API key as an environment variable on your local machine.

Copilot API

1. Visit the Copilot API Documentation: Go to the .

2. Sign Up: Sign up for a Copilot account if you don't already have one.

3. Create API Keys: Go to the Settings > API page and click "Create key" to generate your API key.

4. Set Up Webhooks: If needed, create webhooks on the same page.

Scrape Pinterest Part 1

I am interested in writing a python script to webscrape my pinterest account and pull videos and Gifs from a date range. Pintertest does offer an API that does allow a Python script to do what I want. I even got Microsoft Copilot to generate a working script but using the API is really expensive. Therefore I found another article for that allows you to work around the API. See the following tutorial from Extracting Data from Pinterest using BeautifulSoup, Python. I also hope to use AI to put the downloaded pictures or video in folders for archiving. I wanted to do both in one script but I may have to do a separate script for training a model and organizing files into appropriate folders. Below is code that Microsoft Copilot genereated with a little editing from me.

#Write a python script that uses pinterest api to put pins into the correct board from the last seven days. If there is no board put the pin into the Unknown board.
import datetime, os
try:
    from pinterest_api import Pinterest
except:
    os.system("pip install pinterest-api")
    from pinterest_api import Pinterest
# Initialize Pinterest API
pinterest = Pinterest(access_token='@#$$%#@#')
# Get today's date and the date 7 days ago
today = datetime.datetime.now()
week_ago = today - datetime.timedelta(days=7)
# Get pins from the last 7 days
pins = pinterest.get_user_pins()
# Filter pins from the last 7 days
recent_pins = [pin for pin in pins if week_ago <= datetime.datetime.strptime(pin['created_at'], '%Y-%m-%dT%H:%M:%S') <= today]
# Function to add pin to board
def add_pin_to_board(pin, board_name):
    boards = pinterest.get_boards()
    board = next((board for board in boards if board['name'] == board_name), None)
    
    if board is None:
        # Create 'Unknown' board if it doesn't exist
        if board_name == 'Unknown':
            board = pinterest.create_board(name='Unknown', description='Pins without a specific board')
        else:
            print(f"Board '{board_name}' not found.")
            return
        
    pinterest.create_pin(board_id=board['id'], note=pin['note'], link=pin['link'], image_url=pin['image']['original']['url'])
# Iterate through recent pins and add them to the correct board or 'Unknown'
for pin in recent_pins:
    board_name = pin['board']['name'] if 'board' in pin else 'Unknown'
    add_pin_to_board(pin, board_name)
print("Pins have been organized.")

I did find a No-Code solution That I am planning to explore to make a browser bot:

Scrape Pinterest data without code

Downloading Media from Posts on X

 

We can download media from X using the followinf links

Just paste in the URL

The following tutorial show how to download videos and turn short ones to Gifs.

How to save a GIF from X, formerly known as Twitter

Scrapfly - Web Scraping with BeautifulSoup

Web scraping with a tutorial with Scrapfly

How to Parse Web Data with Python and Beautifulsoup

Scrape X.Com Part 1

I am interested in writing a python script to webscrape X (formerly Twitter) and pull videos and Gifs from my favorite posts. X does offer an API that does allow a Python script to do what I want. I even got Microsoft Copilot to generate a working script but using the API is really expensive. Therefore I found another article for that allows you to work around the API. See the following tutorial from Scrapfly - How to Scrape X.com (Twitter) using Python (2024 Update). I also hope to use AI to put the downloaded pictures or video in folders for archiving. I wanted to do both in one script but I may have to do a separate script for training a model and organizing files into appropriate folders. Below is code that Microsoft Copilot genereated with a little editing from me.

# Prompt
# Write a python script that will download videos and pictures from the bookmarked posts in X and classify them
# according to Python, Books, Comics, and Porn for a given date, creating a folder for each category and putting
# the items in the same matching category and unfavorite the post if it has post with something to download
#
import os
from datetime import datetime
try:
    import tweepy
except:
    os.system("pip install tweepy")
    import tweepy
try:
    import requests
except:
    os.system("pip install requests")
    import requests
# Twitter API credentials
consumer_key = '***********'
consumer_secret = '&&&&&&&&&&&&&&'
access_token = '##########################'
access_token_secret = '@@@@@@@@@@@@@@@@@@@@@'
# Authenticate to Twitter
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
# Directory to save media
base_dir = 'downloaded_media'
if not os.path.exists(base_dir):
    os.makedirs(base_dir)
# Function to download media
def download_media(url, filename):
    response = requests.get(url)
    if response.status_code == 200:
        with open(filename, 'wb') as file:
            file.write(response.content)
# Function to classify media
def classify_media(tweet_text):
    if 'python' in tweet_text.lower():
        return 'Python'
    elif 'book' in tweet_text.lower():
        return 'Books'
    elif 'comic' in tweet_text.lower():
        return 'Comics'
    elif 'porn' in tweet_text.lower():
        return 'Porn'
    else:
        return 'Uncategorized'
# Get bookmarked tweets
bookmarked_tweets = api.get_favorites()  # This gets the liked tweets, adjust as needed for bookmarks
for tweet in bookmarked_tweets:
    tweet_date = tweet.created_at.strftime('%Y-%m-%d')
    tweet_text = tweet.text
    category = classify_media(tweet_text)
    category_dir = os.path.join(base_dir, category, tweet_date)
    if not os.path.exists(category_dir):
        os.makedirs(category_dir)
    
    media = tweet.entities.get('media', [])
    if media:
        for item in media:
            media_url = item['media_url']
            media_type = item['type']
            if media_type in ['photo', 'video']:
                filename = os.path.join(category_dir, os.path.basename(media_url))
                download_media(media_url, filename)
                print(f'Downloaded {media_type} from {media_url} to {category}/{tweet_date}')
        
        # Unfavorite the tweet
        api.destroy_favorite(tweet.id)
        print(f'Unfavorited tweet {tweet.id}')
print('Download complete.')