planeta ai logo
Darmowy szablon automatyzacji

Wyszukiwanie w YouTube RAG z Frontendem przy użyciu Apify, Qdrant i AI

1847
26 dni temu
62
bloków


Własna wyszukiwarka RAG dla filmów z YouTube

Chcesz zbudować własną wyszukiwarkę RAG dla filmów z YouTube? Teraz możesz! Ten szablon n8n pokazuje, jak stworzyć wydajną wyszukiwarkę YouTube, wykorzystującą Apify, Qdrant i wybrany model językowy (LLM), aby szybko i efektywnie przeszukiwać wiele filmów w celach badawczych.

Jak to działa

Proces składa się z dwóch głównych etapów:

  • Etap 1: Pobieranie transkrypcji filmów z YouTube i umieszczanie ich w bazie danych wektorowej. W tym przypadku wykorzystano Apify do scrapowania YouTube i Qdrant do przechowywania osadzeń (embeddings). Transkrypcje są dzielone na mniejsze fragmenty i opatrzone metadanymi, aby ułatwić późniejsze wyszukiwanie i filtrowanie.
  • Etap 2: Budowa interfejsu webowego umożliwiającego użytkownikom przeszukiwanie zwektoryzowanych transkrypcji. Wykorzystano webhook do obsługi prostej aplikacji webowej i API, które dynamicznie pobiera wyniki. Wyszukiwanie wykorzystuje funkcję grupowania wyników Qdrant, która w tym przypadku działa lepiej, zwracając szerszy zakres wyników.

Jak używać

  1. Po skonfigurowaniu wszystkich wymaganych poświadczeń, uruchom kroki 1-3, aby wypełnić magazyn wektorowy danymi.
  2. Następnie aktywuj workflow, aby udostępnić interfejs webowy. Odwiedź URL webhooka w przeglądarce, aby z niego korzystać.
  3. Jeśli korzystasz z szablonu tylko do użytku osobistego, możesz usunąć mechanizm ograniczania liczby żądań w kroku 4.

Wymagania

  • Apify do scrapowania kanałów i filmów na YouTube
  • Qdrant jako magazyn wektorowy
  • OpenAI do modeli językowych i generowania osadzeń

Dostosowywanie szablonu

  • Nie interesują Cię filmy z oficjalnego kanału n8n? Możesz zmienić kanał na dowolny inny - szablon będzie działał pod warunkiem, że filmy nie są prywatne lub zablokowane przed osadzaniem.
  • Technicznie każdy magazyn wektorowy powinien działać, ale może nie oferować tej samej funkcjonalności grupowania wyników. W takim przypadku możesz użyć prostego węzła magazynu wektorowego i wrócić do podstawowego wyszukiwania.

Przykłady zastosowań

Ta automatyzacja może być wykorzystana w wielu różnych scenariuszach, takich jak:

  • Badania rynkowe - szybkie przeszukiwanie treści konkurencji
  • Edukacja - wyszukiwanie konkretnych fragmentów wykładów lub tutoriali
  • Dziennikarstwo - weryfikacja faktów i szukanie wypowiedzi w materiałach wideo
  • Tworzenie treści - wyszukiwanie inspiracji w istniejących materiałach
  • Nauka języków - analiza transkrypcji w obcym języku
  • Customer support - szybkie znajdowanie odpowiedzi w materiałach instruktażowych
  • Analiza trendów - śledzenie pojawiających się tematów w określonych niszach


   Skopiuj kod szablonu   
{"meta":{"instanceId":"408f9fb9940c3cb18ffdef0e0150fe342d6e655c3a9fac21f0f644e8bedabcd9","templateCredsSetupCompleted":true},"nodes":[{"id":"c8221e91-9a9c-489e-a770-7c3cf2cb3328","name":"When clicking ‘Test workflow’","type":"n8n-nodes-base.manualTrigger","position":[-1100,-460],"parameters":{},"typeVersion":1},{"id":"111eccd7-51e5-4b4a-9c30-a69f90397df7","name":"Get Video Subtitles","type":"n8n-nodes-base.httpRequest","position":[540,-360],"parameters":{"url":"https://api.apify.com/v2/acts/streamers~youtube-channel-scraper/run-sync-get-dataset-items","options":{},"jsonBody":"={{n{n "downloadSubtitles": true,n "hasCC": false,n "hasLocation": false,n "hasSubtitles": false,n "is360": false,n "is3D": false,n "is4K": false,n "isBought": false,n "isHD": false,n "isHDR": false,n "isLive": false,n "isVR180": false,n "maxResultStreams": 0,n "maxResults": 1,n "maxResultsShorts": 0,n "preferAutoGeneratedSubtitles": false,n "saveSubsToKVS": false,n "startUrls": [n {n "url": $json.url,n "method": "GET"n }n ],n "subtitlesFormat": "vtt",n "subtitlesLanguage": "en"n}n}}","sendBody":true,"specifyBody":"json","authentication":"genericCredentialType","genericAuthType":"httpHeaderAuth"},"credentials":{"httpHeaderAuth":{"id":"SV9BDKc1cRbZBeoL","name":"Apify.com (personal token)"}},"typeVersion":4.2},{"id":"6dcd5497-89cf-4591-ae79-bd12bbde6256","name":"Chunk Subtitles","type":"n8n-nodes-base.set","position":[740,-360],"parameters":{"options":{},"assignments":{"assignments":[{"id":"fa613ce2-3a2f-42e4-9add-88df00efdb85","name":"vtt","type":"array","value":"={{nArray(n Math.ceil($json.subtitles[0].vtt.length/30_000)n).fill(0)n .map((_,idx) => $json.subtitles[0].vtt.substr(idx*30_000,(idx * 30_000) + 30_000))n}}"}]}},"typeVersion":3.4},{"id":"689fc39e-21d9-4222-9cda-858f21cacc97","name":"Qdrant Vector Store","type":"@n8n/n8n-nodes-langchain.vectorStoreQdrant","position":[1480,-520],"parameters":{"mode":"insert","options":{},"qdrantCollection":{"__rl":true,"mode":"list","value":"n8n_videos","cachedResultName":"n8n_videos"}},"credentials":{"qdrantApi":{"id":"AhUP2CNvcQDRd5au","name":"clients-dev"}},"typeVersion":1},{"id":"ee57ffe2-e3f8-409a-a87a-b69902494598","name":"Default Data Loader","type":"@n8n/n8n-nodes-langchain.documentDefaultDataLoader","position":[1580,-360],"parameters":{"options":{"metadata":{"metadataValues":[{"name":"videoId","value":"={{ $('Video Ref').item.json.id }}"},{"name":"title","value":"={{ $('Video Ref').item.json.title }}"},{"name":"channelId","value":"={{ $('Video Ref').item.json.channelId }}"},{"name":"url","value":"={{ $('Video Ref').item.json.url }}"},{"name":"type","value":"={{ $('Video Ref').item.json.type }}"}]}},"jsonData":"={{ $json.vtt.replaceAll('\n\n', '\n') }}","jsonMode":"expressionData"},"typeVersion":1},{"id":"4ce56ce3-73fc-4ba3-abbf-0401d44ce748","name":"Embeddings","type":"@n8n/n8n-nodes-langchain.embeddingsOpenAi","position":[1460,-360],"parameters":{"options":{}},"credentials":{"openAiApi":{"id":"8gccIjcuf3gvaoEr","name":"OpenAi account"}},"typeVersion":1.2},{"id":"a910822d-073f-42a6-8f6d-5c04dc59fba2","name":"Text Splitter","type":"@n8n/n8n-nodes-langchain.textSplitterRecursiveCharacterTextSplitter","position":[1660,-240],"parameters":{"options":{},"chunkSize":3000},"typeVersion":1},{"id":"0361055c-8fc2-488a-9c43-8e0492648756","name":"For Each Video","type":"n8n-nodes-base.splitInBatches","position":[-400,-360],"parameters":{"options":{}},"typeVersion":3},{"id":"77243da4-38cb-430b-b8f3-2698ec6dd022","name":"Video Ref","type":"n8n-nodes-base.noOp","position":[340,-360],"parameters":{},"typeVersion":1},{"id":"fe63938a-8c17-4191-8fdc-cd044f5de080","name":"For Each Chunk","type":"n8n-nodes-base.splitInBatches","position":[1240,-400],"parameters":{"options":{}},"typeVersion":3},{"id":"4bbbc54a-30b6-4395-bb06-0145dde9abd9","name":"Wait","type":"n8n-nodes-base.wait","position":[1920,-400],"webhookId":"45ae600a-d26f-444b-bbe0-792c5602fe8d","parameters":{"amount":1},"executeOnce":true,"typeVersion":1.1},{"id":"6f581ad9-abc5-4ef7-b2ce-1267ba67999d","name":"Clean Up Output","type":"n8n-nodes-base.set","position":[2100,160],"parameters":{"options":{},"assignments":{"assignments":[{"id":"e9a68e02-1559-4ce5-b338-5259f1030d25","name":"title","type":"string","value":"={{ $json.title }}"},{"id":"3fb553d6-79a1-41f2-8b72-581d667adcea","name":"url","type":"string","value":"=https://www.youtube.com/watch?v={{ $json.videoId }}"},{"id":"9f435790-5b84-4a23-ac39-561c87d0eea1","name":"extract","type":"string","value":"={{ $json.extract.replaceAll('\n', ' ') }}"},{"id":"f3d63c6e-d5c9-40b6-a9a7-ebfe7c6c9a41","name":"timestamp","type":"string","value":"={{n(function(str){n return str.length === 3n ? str[1] + ':' + str[2]n : str.join(':');n})($json.timestamp.split(':'))n}}"},{"id":"f937c492-ddbe-40fe-8e12-a21c21832e4a","name":"videoId","type":"string","value":"={{ $json.videoId }}"},{"id":"2245a7b6-cf9e-4c39-875b-5bf80b1353bc","name":"video_ts","type":"string","value":"={{n(function(timestamp){n const buffer = 5;n let [hr,min,sec] = timestamp.map(x => Number(x));n if (sec === undefined){ sec = min; min = hr; hr = 0 }n return Math.max((hr * 60 * 60) + (min * 60) + sec - buffer, 0);n})($json.timestamp.split(':'))n}}"}]}},"typeVersion":3.4},{"id":"27c19284-1ce6-4756-b385-6433096d4b84","name":"Sort By Video ID","type":"n8n-nodes-base.sort","position":[2280,160],"parameters":{"options":{},"sortFieldsUi":{"sortField":[{"fieldName":"videoId"}]}},"typeVersion":1},{"id":"dc70ab23-8fb0-4f7f-9833-5d6b01407b69","name":"Respond to Webhook","type":"n8n-nodes-base.respondToWebhook","position":[3620,220],"parameters":{"options":{"responseCode":200,"responseHeaders":{"entries":[{"name":"Content-Type","value":"text/html"}]}},"respondWith":"text","responseBody":"=
n
AI Summary
n {{ $json.text }}n
n{{ $json.results }}"},"executeOnce":false,"typeVersion":1.1},{"id":"1c4eac3a-1412-4e24-85fd-359aa065e3db","name":"Extract Results","type":"@n8n/n8n-nodes-langchain.informationExtractor","position":[1080,300],"parameters":{"text":"=n{{ $json.hitsn .map(item => `${item.toJsonString()}`).join('\n')n}}nnn{{ $('Get Query').first().json.query }}n","options":{"systemPromptTemplate":"=Your task is to analyse the collection of video transcripts and extract the parts relevant to the user's query.n* When returning your response, provide from 3 and up to 10 results as extracts of the transcript combined with their timestamps (MM:SS) and include the video title and url.n* When getting the extract for the results, as this will be read by the user, return only the text of the transcript and remove any vtt tags, annotations, markers.n* Refer to the metadata for video title and video url."},"schemaType":"manual","inputSchema":"{n "type": "array",n "items": {n "type":"object",n "required": [n "title","url","extract","timestamp","videoId"n ],n "properties": {n "title": { "type": "string" },n "extract": {n "type": "string",n "description": "extract and transcribe the relevant parts of the transcript removing vtt annotations and markers."n },n "timestamp": { "type": "string" },n "videoId": { "type": "string" }n }n }n}"},"executeOnce":false,"typeVersion":1},{"id":"cc2d88f7-065d-489a-bfc4-cbc13268b59a","name":"SEARCH API","type":"n8n-nodes-base.webhook","position":[-1100,360],"webhookId":"e2768373-66ad-4ad2-948f-c5e278d39595","parameters":{"path":"n8n_videos/api/search","options":{"ignoreBots":false},"responseMode":"responseNode"},"typeVersion":2},{"id":"ae2fb16b-2a31-4616-a2b9-6cd634b64647","name":"Get Query","type":"n8n-nodes-base.set","position":[-360,240],"parameters":{"mode":"raw","options":{},"jsonOutput":"={{n{n query: ((($('SEARCH API').first().json.query?.qn .removeTags())n .removeTags())n .replaceSpecialChars()).substr(0,128),n type: ['video', 'stream'].includes($('SEARCH API').first().json.query.type)n ? $('SEARCH API').first().json.query.typen : undefinedn}n}}"},"typeVersion":3.4},{"id":"6d14cd25-f65a-4162-bcfc-cc2e7e547c23","name":"Generate Template","type":"n8n-nodes-base.set","position":[2460,160],"parameters":{"options":{},"assignments":{"assignments":[{"id":"3e4daf29-906b-4469-b107-6cbc142dab44","name":"results","type":"string","value":"={{n(function(items) {n const groupByVideoId = items.reduce((acc, item) => {n if (!acc[item.videoId]) acc[item.videoId] = [];n acc[item.videoId].push(item);n return acc;n }, {});n n const results = Object.keys(groupByVideoId).map(key => {n const parts = groupByVideoId[key]n .toSorted((a,b) => Number(a.video_ts) - Number(b.video_ts));n const {title,url} = parts[0];n return `
  • n
    n n
    ${title}
    n
    ${url}
    n n
    n ${parts.map(item => `n
    n n [${item.timestamp}]n ...${item.extract}...n n
    `).join('\n')}n
    • `n }).join('\n')nn return `
    ${Object.keys(groupByVideoId).length} Video Result${Object.keys(groupByVideoId).length === 1 ? '' : 's'}
    ${results}`;n})(n $input.all().map(item => item.json).filter(item => item.videoId)n)n}}"}]}},"executeOnce":true,"typeVersion":3.4},{"id":"ccf2744d-3678-40b8-b7a1-2fda1d820dd0","name":"Answer Query","type":"@n8n/n8n-nodes-langchain.chainLlm","position":[2760,180],"parameters":{"text":"={{ $json.results }}nn{{ $('Get Query').first().json.query }}n","messages":{"messageValues":[{"message":"=Using the available results, generate a 1 or 2 sentence answer for the user's query. You may format your answer using markdown."}]},"promptType":"define"},"executeOnce":true,"typeVersion":1.5},{"id":"5c9a1091-1c68-4b83-a002-638236044599","name":"Has Results?","type":"n8n-nodes-base.if","position":[1720,260],"parameters":{"options":{},"conditions":{"options":{"version":2,"leftValue":"","caseSensitive":true,"typeValidation":"strict"},"combinator":"and","conditions":[{"id":"81f8ec3e-be33-469c-8bf5-c3d3575e3764","operator":{"type":"object","operation":"notEmpty","singleValue":true},"leftValue":"={{ $json }}","rightValue":""}]}},"typeVersion":2.2},{"id":"97e6fdf0-db16-4957-bc4f-ce96a8ccc944","name":"Generate Empty Response","type":"n8n-nodes-base.set","position":[1920,360],"parameters":{"options":{},"assignments":{"assignments":[{"id":"7c4bd999-49b1-4532-89ca-c53e98da6b17","name":"text","type":"string","value":"={{ '' }}"},{"id":"916221de-2ec9-4fd9-8029-d7a3de88f395","name":"results","type":"string","value":"=
    0 Video Results
    "}]}},"typeVersion":3.4},{"id":"160933f5-ae39-407d-9124-f24d193f1153","name":"Map Fields","type":"n8n-nodes-base.set","position":[3400,220],"parameters":{"options":{},"assignments":{"assignments":[{"id":"36bb30ce-6baf-492a-ad8c-4bec175e85a0","name":"text","type":"string","value":"={{ $json.data }}"},{"id":"60fb3d9d-0342-41c2-80bb-1899773f4bd7","name":"results","type":"string","value":"={{ $('Generate Template').first().json.results }}"}]}},"typeVersion":3.4},{"id":"ac4881ee-e200-48b9-8339-0a21647cc1c4","name":"Incr Rate Limit","type":"n8n-nodes-base.redis","position":[-900,360],"parameters":{"key":"=n8n_videos_session_{{ $json.headers['x-forwarded-for'] }}","expire":true,"operation":"incr"},"credentials":{"redis":{"id":"zU4DA70qSDrZM1El","name":"Redis account (localhost)"}},"typeVersion":1},{"id":"5aa5880a-d9c4-4263-877c-b7c8b3f36916","name":"10req/min","type":"n8n-nodes-base.if","position":[-720,360],"parameters":{"options":{},"conditions":{"options":{"version":2,"leftValue":"","caseSensitive":true,"typeValidation":"strict"},"combinator":"and","conditions":[{"id":"032093e1-b223-45f4-b1bc-5603134acfe8","operator":{"type":"number","operation":"lt"},"leftValue":"={{ Object.values($json)[0] }}","rightValue":11}]}},"typeVersion":2.2},{"id":"7eb4a8e8-ba35-4a55-98cc-f85395a90604","name":"Vectorise Subworkflow","type":"n8n-nodes-base.executeWorkflow","position":[-180,-360],"parameters":{"mode":"each","options":{"waitForSubWorkflow":true},"workflowId":{"__rl":true,"mode":"id","value":"={{ $workflow.id }}"},"workflowInputs":{"value":{},"schema":[],"mappingMode":"defineBelow","matchingColumns":[],"attemptToConvertTypes":false,"convertFieldsToString":true}},"typeVersion":1.2},{"id":"dd76b922-8522-4d5b-a49b-e3e4cdca333f","name":"Vectorise Subworkflow1","type":"n8n-nodes-base.executeWorkflowTrigger","position":[140,-360],"parameters":{"inputSource":"passthrough"},"typeVersion":1.1},{"id":"7b7c5773-9eac-4356-8d66-793dd9bcc008","name":"WEB UI","type":"n8n-nodes-base.webhook","position":[-1120,920],"webhookId":"8b531d97-c5d7-4a98-8ef0-f19e59cf886f","parameters":{"path":"n8n_videos/","options":{"ignoreBots":true},"responseMode":"responseNode"},"typeVersion":2},{"id":"32a0fb68-1b6f-4d65-8591-f370d0307eb4","name":"Qdrant Groups Search","type":"n8n-nodes-base.httpRequest","position":[0,240],"parameters":{"url":"=http://qdrant:6333/collections/n8n_videos/points/search/groups","method":"POST","options":{},"sendBody":true,"authentication":"predefinedCredentialType","bodyParameters":{"parameters":[{"name":"limit","value":"={{ 4 }}"},{"name":"filter","value":"={{n$('Get Query').first().json.typen ? {n "must": [n {n "key": "metadata.type",n "match": {n "any": [].concat($('Get Query').first().json.type)n }n }n ]n }n : undefinedn}}"},{"name":"with_payload","value":"={{ true }}"},{"name":"group_by","value":"metadata.videoId"},{"name":"group_size","value":"={{ 3 }}"},{"name":"vector","value":"={{ $json.data[0].embedding }}"}]},"nodeCredentialType":"qdrantApi"},"credentials":{"qdrantApi":{"id":"AhUP2CNvcQDRd5au","name":"clients-dev"}},"typeVersion":4.2},{"id":"0ddf6e63-d0db-499d-873c-ea7d82bf1ef6","name":"Get Embeddings","type":"n8n-nodes-base.httpRequest","position":[-180,240],"parameters":{"url":"https://api.openai.com/v1/embeddings","method":"POST","options":{},"sendBody":true,"authentication":"predefinedCredentialType","bodyParameters":{"parameters":[{"name":"input","value":"={{ $json.query }}"},{"name":"model","value":"text-embedding-3-small"}]},"nodeCredentialType":"openAiApi"},"credentials":{"openAiApi":{"id":"8gccIjcuf3gvaoEr","name":"OpenAi account"}},"typeVersion":4.2},{"id":"a2fc2329-79be-49ac-923a-63affdefe464","name":"For Each Group","type":"n8n-nodes-base.splitInBatches","position":[740,220],"parameters":{"options":{}},"typeVersion":3},{"id":"ed300a5f-7eca-4f15-bcc3-7855afed0a9e","name":"Group Ref","type":"n8n-nodes-base.noOp","position":[900,300],"parameters":{},"typeVersion":1},{"id":"4e3b33f6-6274-43f1-8206-adeac3220113","name":"Combine Results","type":"n8n-nodes-base.set","position":[1540,260],"parameters":{"options":{},"assignments":{"assignments":[{"id":"f797071b-f722-4273-a672-932370123fc0","name":"output","type":"array","value":"={{ $input.all().flatMap(item => item.json.output) }}"}]}},"executeOnce":true,"typeVersion":3.4},{"id":"da36d896-3187-40f6-82a5-953ae0fde867","name":"Transcripts to Items","type":"n8n-nodes-base.splitOut","position":[1920,160],"parameters":{"options":{},"fieldToSplitOut":"output"},"typeVersion":1},{"id":"e297fd2c-a551-4efe-8240-26522de4cec3","name":"Respond to Webhook2","type":"n8n-nodes-base.respondToWebhook","position":[-180,460],"parameters":{"options":{"responseCode":200},"respondWith":"text","responseBody":"=
    {{ $json.text }}
    n{{ $json.results }}"},"executeOnce":false,"typeVersion":1.1},{"id":"1cc6136e-b60d-4dc3-aeb7-30efb09b24f1","name":"Respond to Webhook3","type":"n8n-nodes-base.respondToWebhook","position":[2100,360],"parameters":{"options":{"responseCode":200},"respondWith":"text","responseBody":"=
    {{ $json.text }}
    n{{ $json.results }}"},"executeOnce":false,"typeVersion":1.1},{"id":"143115aa-5754-4e43-9bda-332b38cce4ae","name":"Schedule Trigger","type":"n8n-nodes-base.scheduleTrigger","position":[-1100,-280],"parameters":{"rule":{"interval":[{"field":"weeks","triggerAtDay":[6],"triggerAtHour":6}]}},"typeVersion":1.2},{"id":"154adf7a-8390-425c-b96c-641c02c92072","name":"Ignore Already Seen","type":"n8n-nodes-base.removeDuplicates","position":[-620,-360],"parameters":{"options":{},"operation":"removeItemsSeenInPreviousExecutions","dedupeValue":"={{ $json.id }}"},"typeVersion":2},{"id":"c488a718-8c27-4d52-a891-bb875c7b8615","name":"Sticky Note","type":"n8n-nodes-base.stickyNote","position":[-1180,-640],"parameters":{"color":7,"width":1200,"height":560,"content":"## 1. Fetch Latest Videos with [Apify.com](https://www.apify.com?fpr=414q6)n[Learn more about Apify.com](https://www.apify.com?fpr=414q6) - [Youtube Scraper](https://apify.com/streamers/youtube-scraper?fpr=414q6)nnIf you want to save serious time and effort and avoid the low usage limits of the official Youtube API, then you probably want to sign-up for a third-party youtube scraper like the ones found on Apify. Here, I'm using a Youtube Scraper to get the latest videos and livestream recordings from the official n8n channel. Running them through a "remove duplicates" node ensures they aren't processed more than once."},"typeVersion":1},{"id":"09fc0bc7-1efc-495f-b57b-3d62edbe09a6","name":"Get Latest Youtube Videos","type":"n8n-nodes-base.httpRequest","position":[-820,-360],"parameters":{"url":"https://api.apify.com/v2/acts/streamers~youtube-channel-scraper/run-sync-get-dataset-items","options":{},"jsonBody":"={{n{n "maxResultStreams": 10,n "maxResults": 10,n "maxResultsShorts": 0,n "oldestPostDate": "2025-01-01",n "startUrls": [n {n "url": "https://www.youtube.com/@n8n-io",n "method": "GET"n }n ]n}n}}","sendBody":true,"specifyBody":"json","authentication":"genericCredentialType","genericAuthType":"httpHeaderAuth"},"credentials":{"httpHeaderAuth":{"id":"SV9BDKc1cRbZBeoL","name":"Apify.com (personal token)"}},"typeVersion":4.2},{"id":"06384b2c-c30f-457e-94d8-976b187ff633","name":"Sticky Note1","type":"n8n-nodes-base.stickyNote","position":[60,-640],"parameters":{"color":7,"width":860,"height":560,"content":"## 2. Get Video Transcript with [Apify.com](https://www.apify.com?fpr=414q6)n[Learn more about subworkflows](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.executeworkflowtrigger)nn**Note: You won't see this run in editor mode! It runs in the background - see executions tab!**nI've chosen to use a subworkflow to help with performance as processing transcripts into embeddings tends to accumulate a lot of data client side if done otherwise. Here, we're once again using a Youtube scraper on Apify to download the video transcripts - note that technically, these are auto-generated subtitles but are good enough to serve our needs."},"typeVersion":1},{"id":"7a22fa36-955a-4586-8022-bcaa07950621","name":"Sticky Note2","type":"n8n-nodes-base.stickyNote","position":[940,-740],"parameters":{"color":7,"width":1200,"height":660,"content":"## 3. Populate Qdrant Vector Store to Build a Search Indexn[Learn more about Qdrant Vector Store](https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.vectorstoreqdrant)nnUsing Qdrant is a usually my personal preference but for this template, I saw a great use-case for its advanced search APIs which we'll get to later.nDue to the size of the transcripts, we're forced to break them down into smaller chunks to populate our vector store. Note the metadata will be super important for later filtering so best practice, always spend some time to design how you will use your metadata upfront!"},"typeVersion":1},{"id":"063c6f34-0972-4c2e-a3db-d3e3158015b7","name":"Chunks to Items","type":"n8n-nodes-base.splitOut","position":[1020,-400],"parameters":{"options":{},"fieldToSplitOut":"vtt"},"typeVersion":1},{"id":"a088b16e-d518-4746-9f03-6ab0a2a46fc0","name":"Sticky Note3","type":"n8n-nodes-base.stickyNote","position":[-1180,80],"parameters":{"color":7,"width":680,"height":520,"content":"## 4. Search API with Rate Limitingn[Learn more about Redis](https://docs.n8n.io/integrations/builtin/app-nodes/n8n-nodes-base.redis)nnWebhooks are a great feature to build simple APIs with n8n. This particular webhook will serve as your search API which takes user queries as input, searches our vector store of n8n videos and returns a list of matching results as output. A Redis counter can be used as a simple rate limiter to manage resources through feel free to remove if you're publishing the webUI to the public. "},"typeVersion":1},{"id":"3a9d011d-fa8f-4dc0-8e7b-d7e9990f9485","name":"429 Response","type":"n8n-nodes-base.set","position":[-360,460],"parameters":{"options":{},"assignments":{"assignments":[{"id":"7c4bd999-49b1-4532-89ca-c53e98da6b17","name":"text","type":"string","value":"={{ '' }}"},{"id":"916221de-2ec9-4fd9-8029-d7a3de88f395","name":"results","type":"string","value":"=
    Search Limit Reached!
    n
    nSearch requests are limited to 3 per minute.n
    Need more? Download the free template and run it yourself!n
    "}]}},"typeVersion":3.4},{"id":"c4263da3-bc5f-470b-8ace-b03a88b650af","name":"Sticky Note4","type":"n8n-nodes-base.stickyNote","position":[-480,-20],"parameters":{"color":7,"width":1080,"height":680,"content":"## 5. Qdrant Advanced Search - Point Groupsn[Learn more about Qdrant's Search Groups API](https://qdrant.tech/documentation/concepts/search/#search-groups)nnOur goal is to return videos and timestamps within them which are relevant to our user's query. We could using simple similarity search but the problem is your results might be too in-depth on the first and second videos. If we want to increase the breadth ie. cover more videos, then we can use something like Qdrant's Search Group API. This search API uses metadata to group results allowing you to specify a max results limit per video. Unfortunately, as this API is not supported by n8n, we'll have to use the HTTP request node - **Be sure to configure the node's credentials before use!**"},"typeVersion":1},{"id":"5c655b2c-1986-499b-b89d-563e2f755986","name":"Sticky Note5","type":"n8n-nodes-base.stickyNote","position":[640,-20],"parameters":{"color":7,"width":800,"height":640,"content":"## 6. Contextually Understanding Transcripts with AIn[Read more about the Information Extractor](https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.information-extractor)nnOne of the great use-cases for AI/LLM data extraction is that it can save a lot of parsing effort traditional done via code. Here, we provide the raw transcript blocks and the AI will help us pick out only the relevant parts with timestamps which answer our user's query."},"typeVersion":1},{"id":"f825ab4a-b88d-46af-b2f2-271c6b22665c","name":"Sticky Note6","type":"n8n-nodes-base.stickyNote","position":[1460,-20],"parameters":{"color":7,"width":1180,"height":620,"content":"## 7. Generate Results HTML Templaten[Learn more about the Edit Fields node](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.set)nnOnce we have our extracted transcript parts, we just need to re-group the results and build the results list HTML to be rendered back to the user.nI'm using the excellent **htmx** (htmx.org) framework for the Web UI which is perfect for single page applications. The results HTML template generated is heavily influenced by this so it may look strange if you're unfamiliar. Feel free to use whatever is comfortable!"},"typeVersion":1},{"id":"1ed3a232-6c03-4cef-806b-7030615b1e98","name":"Sticky Note7","type":"n8n-nodes-base.stickyNote","position":[2660,-20],"parameters":{"color":7,"width":600,"height":560,"content":"## 8. Summarise Results to Generate Answern[Read more about the Basic LLM node](https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.chainllm)nnThis is a nice extra bit of UX to try and answer the user's query based on the search results. A basic LLM node is perfect for this simple prompt."},"typeVersion":1},{"id":"4d5b1466-8bb8-42e7-9cee-275aa4c905df","name":"Sticky Note8","type":"n8n-nodes-base.stickyNote","position":[3280,-20],"parameters":{"color":7,"width":600,"height":560,"content":"## 9. Return Answer & Search Resultsn[Learn more about the webhook node](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.respondtowebhook/)nnFinally, we combine the AI answer and results list as a HTML response for the Web UI.nPhew, that was a lot to get through! Beyond this template, you can play with different search parameters or approaches to reduce latency, include other content types etc."},"typeVersion":1},{"id":"07edb5ea-a4ad-4b06-91f2-9bdc2d5150a4","name":"Sticky Note9","type":"n8n-nodes-base.stickyNote","position":[-1180,700],"parameters":{"color":7,"width":840,"height":480,"content":"## 10. N8N Video Search Frontend using Web UIn[Learn more about the HTML node](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.html/)nnBuilding and deploying simple webpages using n8n is quite easy and great if you don't want to worry too much about the technical details. Once the n8n template is set to active, you can visit [https:///webhook/n8n_video](/webhook/n8n_video) to use it."},"typeVersion":1},{"id":"8a9593fc-5a7a-4da9-b7d4-a128c0d36e8e","name":"Render Page","type":"n8n-nodes-base.respondToWebhook","position":[-760,920],"parameters":{"options":{"responseCode":200,"responseHeaders":{"entries":[{"name":"Content-Type","value":"text/html"}]}},"respondWith":"text","responseBody":"={{ $json.html }}"},"typeVersion":1.1},{"id":"8fbdd941-fba7-4f89-b216-489a6e80cef9","name":"Generate Webpage","type":"n8n-nodes-base.html","position":[-940,920],"parameters":{"html":"nnnnnN8N Videos Searchnnn nnnn
    n
    n
    n n Search the Official N8N Youtube Channeln Cover a wide range of videos in seconds!n n
    n
    n n
    n
    n
    n n
    n n n n Searchn n
    nn
    n
    n
      n
      n
      n
      n
      n n
      Video will show here.
      n
      n
      n
      n

      About N8N Video Search

      n
      n
      n
      n
      n n n nn"},"typeVersion":1.2},{"id":"054feb14-d740-4d05-9ef5-d5c584e07eb6","name":"Markdown","type":"n8n-nodes-base.markdown","position":[3080,180],"parameters":{"mode":"markdownToHtml","options":{},"markdown":"={{ $json.text }}"},"typeVersion":1},{"id":"76c6fb4b-5b82-4115-8be4-63b6ce652793","name":"Sticky Note10","type":"n8n-nodes-base.stickyNote","position":[-540,880],"parameters":{"color":7,"width":540,"height":420,"content":"**Fig 1. N8N Video Search Frontend**n![screenshot of web frontend](https://res.cloudinary.com/daglih2g8/image/upload/f_auto,q_auto/v1/n8n-workflows/lqd2giei1ap2owjgbuth#full-width)"},"typeVersion":1},{"id":"b8ad1b2b-ff05-46a3-a817-951d02d69b01","name":"Sticky Note11","type":"n8n-nodes-base.stickyNote","position":[2160,-740],"parameters":{"color":5,"width":400,"height":240,"content":"**Create Qdrant Collection**nYou may need to create the qdrant collection manually. Run this in the qdrant dashboard's console.n```nPUT collections/n8n_videosn{n "vectors": {n "distance": "Cosine",n "size": 1536n }n}n```"},"typeVersion":1},{"id":"189d6973-cd09-4176-b04b-4c8feea2e653","name":"Sticky Note12","type":"n8n-nodes-base.stickyNote","position":[-1660,-1160],"parameters":{"width":440,"height":1080,"content":"## Try It Out!n### Ever wanted to build your own RAG search over Youtube videos? Well, now you can! This n8n template shows how you can build a very capable Youtube search engine powered by Apify, Qdrant and your LLM of choice to quickly and efficiently browse over many videos for research or viewing pleasure.nnI originally started to template to ask questions on the "n8n @ scale office-hours" livestream videos but then extended it to include the latest videos on the official channel.nn**Check out a demo here**: [https://jimleuk.app.n8n.cloud/webhook/n8n_videos](https://jimleuk.app.n8n.cloud/webhook/n8n_videos)nn### How it worksn* Stage 1 is to collect the Youtube video transcripts into a vector database. For this, I've used Apify to scrape and Qdrant to store.n* Transcripts are broken down into smaller chunks and carefully tagged with metadata to assist in later search and filtering.n* Stage 2 is to build a web frontend for the user to query the vectorised transcripts. I'm using a webhook to serve a simple web app and API to dynamically fetch the results.n* When searching for a video, I've opted to use Qdrant's search groups API which in this use-case, performs better as it returns a wider range of videos results.n* In the web frontend, when the user clicks on the results, the matching Youtube video plays in an embedded video player.nn### How to usen* Once credentials are all set, first run steps 1 - 3 to populate your vector store.n* Next, set the workflow to active to expose the web frontend. Visit the webhook [https:///webhook/n8n_videos](/webhook/n8n_videos) in your browser to use it.n* If only for personal use, you may want to remove. the rate limiting mechanism in step 4.nn### Customising the templaten* Not interested in official n8n videos? Swap to a different channel.n* Google Gemini may perform better when extracting from transcripts.nn### Need Help?nJoin the [Discord](https://discord.com/invite/XPKeKXeB7d) or ask in the [Forum](https://community.n8n.io/)!"},"typeVersion":1},{"id":"a39f0990-6638-4b07-82ca-e91a9a289dfc","name":"Has Results?1","type":"n8n-nodes-base.if","position":[180,240],"parameters":{"options":{},"conditions":{"options":{"version":2,"leftValue":"","caseSensitive":true,"typeValidation":"strict"},"combinator":"and","conditions":[{"id":"0eb32753-8fd7-4266-a32f-a8f51bf77e93","operator":{"type":"array","operation":"notEmpty","singleValue":true},"leftValue":"={{ $json.result.groups }}","rightValue":""}]}},"typeVersion":2.2},{"id":"2552bce4-587f-45ee-8702-7c919978c87e","name":"Generate Empty Response1","type":"n8n-nodes-base.set","position":[180,460],"parameters":{"options":{},"assignments":{"assignments":[{"id":"7c4bd999-49b1-4532-89ca-c53e98da6b17","name":"text","type":"string","value":"={{ '' }}"},{"id":"916221de-2ec9-4fd9-8029-d7a3de88f395","name":"results","type":"string","value":"=
      0 Video Results
      "}]}},"typeVersion":3.4},{"id":"9e9d74c0-3693-4f6f-b406-0ad9a8592c09","name":"Respond to Webhook4","type":"n8n-nodes-base.respondToWebhook","position":[380,460],"parameters":{"options":{"responseCode":200},"respondWith":"text","responseBody":"=
      {{ $json.text }}
      n{{ $json.results }}"},"executeOnce":false,"typeVersion":1.1},{"id":"e39b205e-1574-4ce6-a87a-a0e090ca3e33","name":"Groups to Items1","type":"n8n-nodes-base.splitOut","position":[380,240],"parameters":{"options":{},"fieldToSplitOut":"result.groups"},"typeVersion":1},{"id":"96e5844a-090b-4e1d-b40a-e0ccf58eac13","name":"OpenAI Chat Model","type":"@n8n/n8n-nodes-langchain.lmChatOpenAi","position":[1080,460],"parameters":{"model":{"__rl":true,"mode":"list","value":"gpt-4o-mini"},"options":{}},"credentials":{"openAiApi":{"id":"8gccIjcuf3gvaoEr","name":"OpenAi account"}},"typeVersion":1.2},{"id":"03c3b930-c677-45e1-974c-8e6fc43acd56","name":"OpenAI Chat Model1","type":"@n8n/n8n-nodes-langchain.lmChatOpenAi","position":[2760,340],"parameters":{"model":{"__rl":true,"mode":"list","value":"gpt-4o-mini"},"options":{}},"credentials":{"openAiApi":{"id":"8gccIjcuf3gvaoEr","name":"OpenAi account"}},"typeVersion":1.2}],"pinData":{},"connections":{"Wait":{"main":[[{"node":"For Each Chunk","type":"main","index":0}]]},"WEB UI":{"main":[[{"node":"Generate Webpage","type":"main","index":0}]]},"Markdown":{"main":[[{"node":"Map Fields","type":"main","index":0}]]},"10req/min":{"main":[[{"node":"Get Query","type":"main","index":0}],[{"node":"429 Response","type":"main","index":0}]]},"Get Query":{"main":[[{"node":"Get Embeddings","type":"main","index":0}]]},"Group Ref":{"main":[[{"node":"Extract Results","type":"main","index":0}]]},"Video Ref":{"main":[[{"node":"Get Video Subtitles","type":"main","index":0}]]},"Embeddings":{"ai_embedding":[[{"node":"Qdrant Vector Store","type":"ai_embedding","index":0}]]},"Map Fields":{"main":[[{"node":"Respond to Webhook","type":"main","index":0}]]},"SEARCH API":{"main":[[{"node":"Incr Rate Limit","type":"main","index":0}]]},"429 Response":{"main":[[{"node":"Respond to Webhook2","type":"main","index":0}]]},"Answer Query":{"main":[[{"node":"Markdown","type":"main","index":0}]]},"Has Results?":{"main":[[{"node":"Transcripts to Items","type":"main","index":0}],[{"node":"Generate Empty Response","type":"main","index":0}]]},"Has Results?1":{"main":[[{"node":"Groups to Items1","type":"main","index":0}],[{"node":"Generate Empty Response1","type":"main","index":0}]]},"Text Splitter":{"ai_textSplitter":[[{"node":"Default Data Loader","type":"ai_textSplitter","index":0}]]},"For Each Chunk":{"main":[[],[{"node":"Qdrant Vector Store","type":"main","index":0}]]},"For Each Group":{"main":[[{"node":"Combine Results","type":"main","index":0}],[{"node":"Group Ref","type":"main","index":0}]]},"For Each Video":{"main":[[],[{"node":"Vectorise Subworkflow","type":"main","index":0}]]},"Get Embeddings":{"main":[[{"node":"Qdrant Groups Search","type":"main","index":0}]]},"Chunk Subtitles":{"main":[[{"node":"Chunks to Items","type":"main","index":0}]]},"Chunks to Items":{"main":[[{"node":"For Each Chunk","type":"main","index":0}]]},"Clean Up Output":{"main":[[{"node":"Sort By Video ID","type":"main","index":0}]]},"Combine Results":{"main":[[{"node":"Has Results?","type":"main","index":0}]]},"Extract Results":{"main":[[{"node":"For Each Group","type":"main","index":0}]]},"Incr Rate Limit":{"main":[[{"node":"10req/min","type":"main","index":0}]]},"Generate Webpage":{"main":[[{"node":"Render Page","type":"main","index":0}]]},"Groups to Items1":{"main":[[{"node":"For Each Group","type":"main","index":0}]]},"Schedule Trigger":{"main":[[{"node":"Get Latest Youtube Videos","type":"main","index":0}]]},"Sort By Video ID":{"main":[[{"node":"Generate Template","type":"main","index":0}]]},"Generate Template":{"main":[[{"node":"Answer Query","type":"main","index":0}]]},"OpenAI Chat Model":{"ai_languageModel":[[{"node":"Extract Results","type":"ai_languageModel","index":0}]]},"OpenAI Chat Model1":{"ai_languageModel":[[{"node":"Answer Query","type":"ai_languageModel","index":0}]]},"Default Data Loader":{"ai_document":[[{"node":"Qdrant Vector Store","type":"ai_document","index":0}]]},"Get Video Subtitles":{"main":[[{"node":"Chunk Subtitles","type":"main","index":0}]]},"Ignore Already Seen":{"main":[[{"node":"For Each Video","type":"main","index":0}]]},"Qdrant Vector Store":{"main":[[{"node":"Wait","type":"main","index":0}]]},"Qdrant Groups Search":{"main":[[{"node":"Has Results?1","type":"main","index":0}]]},"Transcripts to Items":{"main":[[{"node":"Clean Up Output","type":"main","index":0}]]},"Vectorise Subworkflow":{"main":[[{"node":"For Each Video","type":"main","index":0}]]},"Vectorise Subworkflow1":{"main":[[{"node":"Video Ref","type":"main","index":0}]]},"Generate Empty Response":{"main":[[{"node":"Respond to Webhook3","type":"main","index":0}]]},"Generate Empty Response1":{"main":[[{"node":"Respond to Webhook4","type":"main","index":0}]]},"Get Latest Youtube Videos":{"main":[[{"node":"Ignore Already Seen","type":"main","index":0}]]},"When clicking ‘Test workflow’":{"main":[[{"node":"Get Latest Youtube Videos","type":"main","index":0}]]}}}
      Planeta AI 2025 
      magic-wandmenu linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram