Guide to Google News API and Alternatives
Discover how to access Google News after the discontinuation of the Google News API. Explore alternative APIs for extracting insights from news.
TikTok has rapidly grown into one of the most popular social media platforms, attracting users and businesses alike. As the platform’s reach expands, so does the demand for data and insights.
There are several official TikTok’s APIs which provide developers with tools to integrate TikTok functionalities into their applications and access TikTok data. This guide explores the available TikTok APIs, their use cases, and alternative methods for obtaining TikTok data.
This tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect and here's a good summary of what not to do:
Scrapfly does not offer legal advice but these are good general rules to follow in web scraping
and for more you should consult a lawyer.
TikTok offers several APIs designed to serve different needs. Below, each API is explained in detail:
The TikTok Login Kit allows users to log in to third-party applications using their TikTok credentials. This API simplifies authentication and helps developers personalize user experiences by integrating TikTok accounts seamlessly.
Features:
Use Case:
An e-commerce app uses the TikTok Login Kit to allow users to log in with their TikTok credentials. This streamlines registration and enables the app to offer personalized recommendations based on the user's TikTok activity.
The TikTok Share Kit enables users to share content from third-party apps directly to TikTok. It supports sharing videos, hashtags, captions, and more, making content creation and engagement more fluid.
Features:
Use Case:
A video editing app integrates the Share Kit to allow users to post their edited videos to TikTok with pre-filled trending hashtags and captions, increasing user engagement and app retention.
The The Content Posting API provides functionality for developers to automate the uploading of videos to TikTok. This API is particularly useful for businesses managing multiple accounts or scheduling content.
Features:
Use Case:
A digital marketing agency uses the Content Posting API to manage TikTok campaigns for multiple clients, ensuring posts are published during peak engagement times.
The The Data Portability API facilitates the transfer of user data between TikTok and third-party applications, ensuring compliance with data privacy regulations and enhancing user control over their information.
Features:
Use Case:
A fitness app allows users to import their TikTok activity data through the Data Portability API to create customized workout challenges based on trending TikTok fitness trends.
The The Display API allows developers to get basic TikTok profile info and content, such as videos and user feeds. This API is ideal for showcasing TikTok trends and enhancing content discoverability.
Features:
Use Case:
A news website displays their trending TikTok videos on its homepage using the Display API, providing readers with engaging, real-time multimedia content.
TikTok’s Research API is a specialized tool for academic and market research. It enables researchers to access anonymized and aggregated data for studying user behavior, trends, and platform dynamics.
Features:
Use Case:
A university research team uses the Research API to study the impact of TikTok challenges on adolescent mental health, utilizing anonymized data to maintain privacy.
The Commercial Content API is considered part for the research tools tiktok offer. It provides access to public advertiser data.
For example, you can query the TikTok ads created in Italy between January 2, 2021 to January 9, 2021 with the keyword "coffee".
The TikTok Business API is a robust tool for brands and advertisers. It allows businesses to create, manage, and optimize ad campaigns, providing comprehensive tools for audience targeting and analytics.
Features:
Use Case:
An e-commerce retailer uses the TikTok Business API to launch a series of targeted ad campaigns, optimizing for conversions based on real-time performance data.
However, this API is strictly geared towards business-related functionalities. It does not provide access to raw public tiktok data. For those needs, researchers often turn to the TikTok Research API.
For those unable to access TikTok’s official Research API, web scraping can serve as an alternative. This method involves extracting data directly from TikTok’s publicly accessible web pages using automated scripts. While effective, it comes with several caveats:
At scrapfly, we are dedicated to provide developer withs all the resources they need to reach their scraping goals. Check out our comprehensive guide on scraping tiktok as well as our example tiktok scraper using Scrapfly's APIs on github.
Since TikTok employs anti-scraping mechanisms to prevent web-scrapers from accessing tiktok data, scraping tiktok with the traditional approaches cannot be done.
ScrapFly provides web scraping, screenshot, and extraction APIs for data collection at scale.
Here is simple python code the uses Scrapfly's python SDK to scrape tiktok posts and parse them into JSON data:
import json
import jmespath
from typing import Dict, List
from urllib.parse import urlencode, quote, urlparse, parse_qs
from loguru import logger as log
from scrapfly import ScrapeConfig, ScrapflyClient, ScrapeApiResponse
SCRAPFLY = ScrapflyClient(key="YOUR_SCRAPFLY_KEY")
BASE_CONFIG = {
# bypass tiktok.com web scraping blocking
"asp": True,
# set the proxy country to US
"country": "US",
}
def parse_post(response: ScrapeApiResponse) -> Dict:
"""parse hidden post data from HTML"""
selector = response.selector
data = selector.xpath("//script[@id='__UNIVERSAL_DATA_FOR_REHYDRATION__']/text()").get()
post_data = json.loads(data)["__DEFAULT_SCOPE__"]["webapp.video-detail"]["itemInfo"]["itemStruct"]
parsed_post_data = jmespath.search(
"""{
id: id,
desc: desc,
createTime: createTime,
video: video.{duration: duration, ratio: ratio, cover: cover, playAddr: playAddr, downloadAddr: downloadAddr, bitrate: bitrate},
author: author.{id: id, uniqueId: uniqueId, nickname: nickname, avatarLarger: avatarLarger, signature: signature, verified: verified},
stats: stats,
locationCreated: locationCreated,
diversificationLabels: diversificationLabels,
suggestedWords: suggestedWords,
contents: contents[].{textExtra: textExtra[].{hashtagName: hashtagName}}
}""",
post_data,
)
return parsed_post_data
async def scrape_posts(urls: List[str]) -> List[Dict]:
"""scrape tiktok posts data from their URLs"""
to_scrape = [ScrapeConfig(url, **BASE_CONFIG) for url in urls]
data = []
async for response in SCRAPFLY.concurrent_scrape(to_scrape):
post_data = parse_post(response)
data.append(post_data)
log.success(f"scraped {len(data)} posts from post pages")
return data
To wrap up our intro to official Tiktok API here are some frequently asked questions that we might have not covered in the article:
Typically, no. Access is restricted to accredited researchers or organizations that meet specific criteria.
Accessing TikTok APIs typically requires developers to create an account on the TikTok Developers Portal and register their application. Some APIs, like the Research API, require additional approval, which includes meeting criteria such as being an accredited researcher or organization, demonstrating a legitimate purpose, and adhering to privacy guidelines.
Most TikTok APIs, such as the Business API and Research API, are not designed to provide raw public data. Access to public data via the Research API is limited to approved researchers and comes with strict data privacy and usage guidelines. Developers looking for public data may consider alternatives like web scraping.
TikTok’s APIs offer a range of tools for developers, businesses, and researchers, but access and functionality are often limited by stringent requirements. For those looking to extract large-scale data, the Research API is the most suitable option but comes with access restrictions.
Web scraping remains an alternative, albeit with significant risks and limitations. Understanding these tools and their boundaries is essential for making informed decisions about TikTok data integration and analysis.