Skip to main content

Data Source

Follow the instructions below to create a new data flow that ingests data from a News API source in Nexla.
newsapi_api.png

News API

Create a New Data Flow​

  1. To create a new data flow, navigate to the Integrate section, and click the New Data Flow button. Then, select the desired flow type from the list, and click the Create button.

  2. Select the News API connector tile from the list of available connectors. Then, select the credential that will be used to connect to your News API account, and click Next; or, create a new News API credential for use in this flow.

  3. In Nexla, News API data sources can be created using pre-built endpoint templates, which expedite source setup for common News API endpoints. Each template is designed specifically for the corresponding News API endpoint, making source configuration easy and efficient.
    • To configure this source using a template, follow the instructions in Configure Using a Template.

    News API sources can also be configured manually, allowing you to ingest data from News API endpoints not included in the pre-built templates or apply further customizations to exactly suit your needs.
    • To configure this source manually, follow the instructions in Configure Manually.

Configure Using a Template​

Nexla provides pre-built templates that can be used to rapidly configure data sources to ingest data from common News API endpoints. Each template is designed specifically for the corresponding News API endpoint, making data source setup easy and efficient.

Endpoint Settings​

  • Select the endpoint from which this source will fetch data from the Endpoint pulldown menu. Available endpoint templates are listed in the expandable boxes below. Click on an endpoint to see more information about it and how to configure your data source for this endpoint.

    Get Breaking News Headlines For a Country

    This endpoint template fetches top and breaking headlines for a specific category in a country from the News API. Use this template when you need to retrieve current news headlines filtered by country and category, such as business news from the US or technology news from the UK.

    • Enter the country code in the Country (or 2 letter code) field. This should be the 2-letter ISO 3166-1 country code (e.g., us for United States, gb for United Kingdom, ca for Canada). You can select from the dropdown or enter a custom 2-letter country code. The country code determines which country's headlines will be retrieved.
    • Enter the category code in the Category field. Available categories include business, entertainment, general, health, science, sports, and technology. You can select from the dropdown or enter a custom category code. The category determines which type of headlines will be retrieved.

    This endpoint retrieves top and breaking headlines for the specified country and category. The endpoint uses incrementing page-based pagination to handle large datasets efficiently. Nexla will automatically fetch subsequent pages of data by incrementing the page number. The endpoint returns up to 100 articles per page by default.

    Country codes must be valid 2-letter ISO 3166-1 codes. Category codes must be valid News API category values. For detailed information about available countries, categories, pagination, and API usage limits, see the News API top headlines documentation.

    Get Breaking News Headlines by Keyword Search

    This endpoint template fetches top and breaking headlines that match a keyword search from the News API. Use this template when you need to retrieve current news headlines filtered by keywords, such as headlines about "artificial intelligence" or "climate change". Articles are sorted by the earliest date published first.

    • Enter your search query in the Query String field. This should be the keyword or phrase you want to search for in headlines (e.g., artificial intelligence, climate change, technology). The query string is used to filter headlines that match your search criteria.

    This endpoint retrieves top and breaking headlines that match the keyword search. The endpoint uses incrementing page-based pagination to handle large datasets efficiently. Nexla will automatically fetch subsequent pages of data by incrementing the page number. The endpoint returns up to 100 articles per page by default. Articles are sorted by the earliest date published first.

    For detailed information about query syntax, pagination, and API usage limits, see the News API top headlines documentation.

    Search for Articles by Keyword/Query

    This endpoint template searches for articles using a keyword or query string from the News API. Use this template when you need to search across all available articles (not just top headlines) for specific keywords or topics, such as searching for all articles about "machine learning" or "renewable energy".

    • Enter your search query in the Query String field. This should be the keyword or phrase you want to search for in articles (e.g., machine learning, renewable energy, startup funding). The query string is used to filter articles that match your search criteria.

    This endpoint searches across all available articles (not just top headlines) for articles matching the keyword search. The endpoint uses incrementing page-based pagination to handle large datasets efficiently. Nexla will automatically fetch subsequent pages of data by incrementing the page number. The endpoint returns up to 100 articles per page by default.

    For detailed information about query syntax, pagination, date filtering, and API usage limits, see the News API everything endpoint documentation.

Endpoint Testing​

Once the selected endpoint template has been configured, Nexla can retrieve a sample of the data that will be fetched according to the current settings. This allows users to verify that the source is configured correctly before saving.

  • To test the current endpoint configuration, click the Test button to the right of the endpoint selection menu. Sample data will be fetched & displayed in the Endpoint Test Result panel on the right.

  • If the sample data is not as expected, review the selected endpoint and associated settings, and make any necessary adjustments. Then, click the Test button again, and check the sample data to ensure that the correct information is displayed.

Configure Manually​

News API data sources can be manually configured to ingest data from any valid News API endpoint. Manual configuration provides maximum flexibility for accessing endpoints not covered by pre-built templates or when you need custom API configurations.

With manual configuration, you can also create more complex News API sources, such as sources that use custom query parameters or filters.

API Method​

  1. To manually configure this source, select the Advanced tab at the top of the configuration screen.

  2. Select the API method that will be used for calls to the News API from the Method pulldown menu. News API typically uses GET method for retrieving data from endpoints.

API Endpoint URL​

  1. Enter the URL of the News API endpoint from which this source will fetch data in the Set API URL field. This should be the complete URL including the protocol (https://), the API base URL (newsapi.org/v2), the endpoint path (e.g., /top-headlines or /everything), and any required query parameters (e.g., https://newsapi.org/v2/top-headlines?country=us&category=business). Include any required query parameters in the URL.

Ensure the API endpoint URL is correct and accessible with your current credentials. The News API uses incrementing page-based pagination for some endpoints. You can test the endpoint using the Test button after configuring the URL.

Response Data Path​

  1. Enter the JSONPath expression in the Response Data Path field to specify which part of the API response should be treated as the relevant data by Nexla. For News API responses, use $.articles[*] to extract all articles from the articles array in the response.

The JSONPath expression must correctly reference the structure of your News API response. News API responses typically have an articles property containing an array of article objects. Ensure your JSONPath expression matches the structure returned by your specific endpoint. The JSONPath expression determines which data will be extracted and processed by Nexla.

Pagination (if applicable)​

  1. If your endpoint supports pagination, configure the pagination settings in the Pagination section. News API uses incrementing page-based pagination for most endpoints. Select the pagination type that matches your endpoint's pagination mechanism.

  2. Configure the pagination parameters based on your selected pagination type. For incrementing page-based pagination, specify the page parameter name (page), the starting page number (1), and the page size parameter name (pageSize).

Save the Data Source​

  1. Once all of the relevant steps in the above sections have been completed, click the Next button to proceed with the rest of the data flow configuration, or click Save to save the data source configuration for later use.