Difference between revisions of "Google Summer of Code/2020"

From Official Kodi Wiki
Jump to: navigation, search
(Game-theoretic engine for RetroPlayer)
(Game-theoretic engine for RetroPlayer)
 
(One intermediate revision by the same user not shown)
Line 240: Line 240:
 
===Game-theoretic engine for RetroPlayer===
 
===Game-theoretic engine for RetroPlayer===
  
'''Description''': Currently, Kodi's game engine (RetroPlayer) is based on VideoPlayer - games act as "codecs" that accept input and produce audio and video. While this rocks for performance (we've even mirrored zero-copy support from VP), it makes many features, such as rewind, runahead, and netplay difficult to integrate.
+
'''Description''': Currently, Kodi's game engine (RetroPlayer) is based on VideoPlayer - games act as "codecs" that accept input and produce audio and video streams. While this rocks for performance (we've even mirrored zero-copy support from VP), it makes many features, such as rewind, runahead, and netplay difficult to integrate.
  
  
Line 288: Line 288:
  
 
'''Difficulty''': hell
 
'''Difficulty''': hell
 +
 +
'''Type''': Fun/Peripheral
 +
 +
===Retroverse===
 +
 +
'''Description''': This project introduces the Retroverse: an infinitely-rewindable branching history of all gameplay.
 +
 +
Braid creator Jonathan Blow gave a presentation titled [https://www.youtube.com/watch?v=8dinUbg2h70 The Implementation of Rewind in Braid]. In 2013, Themaister [https://github.com/garbear/xbmc-retroplayer/commit/97e52df8a2f82a36952c8bcbebc133b3efc1babe donated] braid-style rewind to Kodi. This became the basis of the [https://github.com/xbmc/xbmc/blob/master/xbmc/cores/RetroPlayer/streams/memory/IMemoryStream.h memory streaming API] in RetroPlayer.
 +
 +
Recall how Blow describes his rewind feature: "Record full world state for every frame always; do not drop frames. Compress data somehow." The challenge of this project is to figure out that "somehow".
 +
 +
Does recording hundreds of hours of gameplay sound impossible? Assume deterministic emulation. The State can be fully reconstructed from the Action of the previous timestep, so only the Action needs to be saved. The Atari 2600, released in 1977, has 5 buttons, needing 5 bits. With a 5 bit frame size, you can store around 850 years of continuous (or nonlinear) Atari 2600 gameplay on a 1TB drive. Still seem impossible?
 +
 +
'''Expected Outcome''': Implement the <code>CNonlinearMemoryStream</code> class referenced in <code>[https://github.com/xbmc/xbmc/blob/master/xbmc/cores/RetroPlayer/streams/memory/IMemoryStream.h IMemoryStream.h]</code>. Enable LZ4 and/or RLE compression of savestate diffs. Enhance autosave by using interpolation to drop "base frames". Store a tree instead of a buffer!
 +
 +
'''Skills preferred''': C++
 +
 +
'''Possible mentor''': garbear, velocity
 +
 +
'''Difficulty''': advanced
  
 
'''Type''': Fun/Peripheral
 
'''Type''': Fun/Peripheral

Latest revision as of 07:43, 11 February 2020

Home icon grey.png   ▶ Google Summer of Code ▶ 2020
GSOC 2016 logo.png
Welcome to the Kodi Google Summer of Code ideas page.

We encourage interested students of all ethnities and genders to review some of the ideas on this page, and then feel free to provide input on any ideas you may have in the Kodi GSoC 2020 forum and chat about any project you’d love to cover. To propose a project, see #Students project proposal ideas.

From the 25th March to the 9th of April, any interested students may apply at the GSOC home page to work with Kodi. After that, Google will notify applicants whether we get to work with each other according to the GSOC schedule.

1 About Us

As there may be many students who have landed here as a result of GSOC, but are unfamiliar with the project, here are a few resources that may help explain what we are about.

Kodi (formerly known as XBMC) is an award-winning free and open source (GPL) software media player and entertainment hub for digital media, designed around the 10-foot interface (living room) environment. Created in 2003 by a group of like minded programmers, Kodi is a non-profit project run and developed by volunteers located around the world. More than 500 software developers have contributed to Kodi, and 100-plus translators have worked to expand its reach, making it available in more than 60 languages. For more information, see the page Kodi.


To get an idea of what Kodi is truly capable of, it really must be seen. Check out a few other user-created videos:
Kodi with the default Confluence skin
Kodi with Aeon Nox skin
Kodi on Raspberry Pi
Kodi's new PVR functionality

Kodi is written primarily in c++ and runs on a variety of platforms including Android, iOS, Linux, OS X, and Windows. It has been ported to work on several low-power platforms including the Raspberry Pi and Android.

Kodi was a mentoring organization in 2008, 2012, 2013, 2015, 2017, 2018, 2019 and had team members involved in GSoC for other projects during 2011.

If Kodi is selected as a mentoring organization for 2020, students will need to review the Overview of a good project proposal, follow the outline for proposals when applying, and review the list of project ideas detailed below. Students are welcome to propose ideas outside the list and are encouraged to be as creative as they like.

1.1 Mentors

All mentors and backup mentors are extremely experienced in the Kodi codebase and will thus be able to assist students in getting to know the codebase and in quickly identifying projects that are both achievable for someone unfamiliar with the internal workings of Kodi and desirable to the wider Kodi community.

2 Prerequisites

C++ coding skills, basic familiarity with Git, solid understanding and interest in programming. Ability to quickly understand existing code is beneficial.

3 Project Proposals

3.1 Overview

Qualifications for a good Summer of Code proposal:

  • Discrete, well-defined, modular
  • Comprised of a series of measurable sub-goals
  • Based on open specs that are available free of charge
  • Based on complete specs

An example of a good proposal is the implementation of a new feature or function that is not yet available in Kodi.

An example of a less desirable proposal is one that's not as measurable, such as refactoring an existing API. Bad proposals tend to be ones that would require touching a lot of core code.

To re-iterate:

  • Localized/isolated code projects = good
  • Global code refactoring = bad
  • A project should have a set of subgoals, so even if the end goal turns out to be too big some of the parts will be of benefit.
  • Not too big! This is an important problem when choosing a project, while it is fun to think about solving a grand project its not always realistic. Its better to finish a smaller project than to start a grand one.

3.2 Where to submit proposals

In addition to submitting to the Google Summer of Code website, you are highly encouraged to submit your idea/proposal to the Kodi forum for discussion. Any proposal not submitted to the forum for discussion will likely not be considered.

3.3 Outline for proposals

4 Team-Kodi proposal ideas up for discussion

4.1 Explore building binary addons in rust

Description: While Kodi has python addons, it can also handle binary addons. But these need to be build one by one for each platform and also might fail on errors. Rust might help with both of those problems and more. The idea is to use the api we have but via rust ffi and see where that leads us.

Expected outcome: Documentation and templates on how to use rust with kodi. In the best case also one or two example addons.

Skills preferred: C++, Rust, C

Possible mentors: spiff, Razze, velocity

Difficulty: Hard

Type: Binary addon development

4.2 A new web interface

Description: Kodi has a webinterface by the name of chorus2 (https://github.com/xbmc/chorus2) which is written in coffeescript and as it stands no team member is able to work on coffeescript. So the idea would be to implement a new interface, in a newer language. I would like to see Elm or Svelte, but others would be fine too, as long as we would be able to maintain it.

Expected outcome: A new webinterface (can be redesigned) or a port of the old one.

Skills preferred: HTML, CSS, Javascript, Typescript, Elm

Possible mentors: Razze

Difficulty: Medium

Type: Webinterface development

4.3 Better profile support

Description: Profile support inside Kodi remains extremely hacked together. Improving profile projects would likely be broken down into smaller chunks to make each chunk truly achievable in a summer. Database handling for profiles needs to be reviewed and changed so that the user experience is more streamlined. It should also be possible to better guard code paths that need to check for specific profile rights.

Expected outcome: Profiles should be more user-friendly than before and not be an afterthought. Being able to easily add more profile features is the big ultimate goal, as we want to support profiles for kids that automatically hide media that's not for them. Just as an example.

Skills preferred: C++

Possible mentors: spiff

Difficulty: Hard

Type: Core development

4.4 Replacing depends with a CMake-based system

Description: Kodi has its own system for building the many libraries Kodi depends on that supports most platforms (Linux, Android, OS X, iOS) and cross-compiling called depends. At the moment, it is implemented with autotools and hand-crafted Makefiles. This has lead to a lot of code duplication, poor maintainability, and not being very user-friendly. Also, it does not currently work on Windows. This task would be about replacing the current system with a new implementation in CMake that has better maintainability.

Expected outcome: A CMake-based dependency build system that offers roughly the same features as depends (i.e. all required libraries covered, diverse platform support, crosscompilation). If the solution can also be applied to Windows by e.g. adding minimal CMake files to replace some UNIX-only build systems, that would be a big plus, but it is not necessary.

Skills preferred: CMake, shell scripting, general familiarity with UNIX/Linux

Possible mentors: wsnipex

Difficulty: Medium

Type: Infrastructure/Automation

4.5 Integrate TUF (The Update Framework)

Description: Kodi uses a self-built mechanism for installing and updating add-ons from add-on repositories. Unfortunately, it is not very secure. This project would be about replacing the add-on repository code with the usage of TUF (The Update Framework), a quite recent library that solves many common security problems for updaters. TUF does not currently have a C or C++ implementation. The reference implementation is in Python, which we do not want to integrate with C++ for this core piece of application code. Due to security concerns, we also do not want to implement TUF ourselves. That leaves using rust-tuf, an experimental Rust implementation. First step would be to write a C wrapper library so that we can use it from Kodi.

Expected outcome: Add-on repository in Kodi using TUF that builds and works on all supported platforms (beware, there are quite a lot) and is integrated into the Kodi add-on infrastructure (i.e. there are also tools to generate valid repositories). It should support all core add-on functionality (be able to list, install, and update add-ons).

Skills preferred: C++, Rust, Python (for creating the repository the TUF reference implementation could be used)

Possible mentors: yol

Difficulty: Hard

Type: Risky/Exploratory (rust-tuf is unstable and there is also a lot of work to be done on the backend side)

4.6 Add-ons

4.6.1 Automatic add-on checker for binary add-ons

Description: Kodis add-on checker checks each Kodi add-on PR on GitHub for certain problems, which ultimately makes reviewing Python add-ons easier for the team. But the other type of add-ons, binary add-ons is not being checked so far, which makes reviews and coding guidelines hard to handle. Thus it would be nice to also check those add-ons, either via the current checker written in Python or a new binary add-on specific checker. Goals would include checking the addon.xml, the C++ code and a check for potentially malicious code.

Expected outcome: Being able to run the checker against all binary repo PRs on GitHub.

Skills preferred: Python, C++

Possible mentors: Razze

Difficulty: Hard

Type: Low-hanging fruit

4.7 Achievements in RetroPlayer

Description: Retroarch has developed support for achievements in certain select libretro cores. This project would consist of porting whatever work might be portable and otherwise integrating the existing libretro achievement system into Kodi's RetroPlayer implemented using Kodi's UI system. Could be tackled a few different ways, including simply linking the user to the retroachievements.org or creating a full achievement database in Kodi.

Expected Outcome: Users should be able to see their achievements in Kodi. Player manager and Player profile support may be in development concurrently, so thought should be given to those systems.

Skills preferred: C++, python, possibly php

Possible mentor: garbear, velocity

Difficulty: medium

Type: Fun/Peripheral

Notes: Garbear has broken down the Retroplayer project into manageable pieces including input, core, game add-ons, peripheral add-ons, netplay, a game library implementation, and shaders to do things like emulate a CRT monitor. This is one example, but many pieces are currently being worked on or are near completion, and many more could use help from a student familiar with the area. Visit the retroplayer section of the Kodi forum to see all the major projects still to do.

4.8 Saved game manager for RetroPlayer

Description: Kodi excels at managing large movie, music, and (with add-ons) game libraries. However, savestates and in-game saves are simply stored next to the ROM or in a hard-coded folder.

This project would consist of designing a user interface for saved game management. Some database work will probably be required. As a stretch goal, automatic saved-game captioning can be performed using the "Rich Presence" feature of RetroAchievements.org.

In 2016, Garbear created an experimental saved game manager - see https://github.com/xbmc/xbmc/pull/11034. It may be possible to re-use ideas and code from this PR. Here's what the saved game manager looked like at the time:

Saved game manager from 2016

Expected Outcome: Users should be able to manage their savestates and in-game saves in Kodi.

Skills preferred: C++, optional python

Possible mentor: garbear, velocity

Difficulty: medium

Type: Fun/Peripheral

4.9 Runahead for RetroPlayer

Description: This feature, pioneered by libretro (see Achieving better latency than original hardware through new runahead method) allows for input latency better than the original console!

Expected Outcome: Implemented runahead input

Skills preferred: C++

Possible mentor: garbear, velocity

Difficulty: medium

Type: Fun/Peripheral

4.10 Player Manager for RetroPlayer

Description: Currently, player assignment and controller selection is hard-coded. This project would allow the user to configure which controllers are connected to the virtual console, enabling multiplayer in games that are currently hard-coded to single player.

A player management concept is outlined here: https://github.com/garbear/xbmc/issues/87. Ideas and code from the issue can possibly be re-used in this project.

Player manager concept art

Expected Outcome: Players can change virtual input devices and choose which player number they are

Skills preferred: C++

Possible mentor: garbear, velocity

Difficulty: medium

Type: Fun/Peripheral

4.11 <insert feature from RetroArch here>

Description: RetroArch is an amazing, full-featured emulator system. Many of these features would make good GSoC projects!

Expected Outcome: Feature from RetroArch is implemented

Skills preferred: C++, optional Python

Possible mentor: garbear, velocity

Difficulty: various

Type: Fun/Peripheral

4.12 Game-theoretic engine for RetroPlayer

Description: Currently, Kodi's game engine (RetroPlayer) is based on VideoPlayer - games act as "codecs" that accept input and produce audio and video streams. While this rocks for performance (we've even mirrored zero-copy support from VP), it makes many features, such as rewind, runahead, and netplay difficult to integrate.


The project is to implement the emulation equation described by the Special Theory of Emulation introduced in Reinforcement Learning Bots, where S is State and A is Action:


Initial value of S
Initial value of A


State update
Action update


The equation results in the time series:


(S0, A0, S1, A1, S2, A2, ...)


The emulation equation describes all of emulation in a game-theoretic manner. With such an engine, powerful features become incredibly easy:

  • When emulation is a time series, rewind is just decrementing t
  • Runahead is just replacing an A several t's ago
  • Netplay is also possible - just run a consensus algorithm like RAFT over the State/Action pairs.


Expected Outcome: Games play using the new engine. Rewind is now easy. Runahead is now easy. Netplay is still hard, but the emulation equation turns netplay from a networking problem into a consensus problem.

Skills preferred: C++

Possible mentor: garbear, velocity

Difficulty: advanced

Type: Fun/Peripheral

4.13 Reinforcement learning bots for RetroPlayer

Description: Same as the above, but implement the emulation equation described by the General Theory of Emulation in Reinforcement Learning Bots.

Expected Outcome: Super-human gaming bots take over the world

Skills preferred: C++

Possible mentor: garbear, velocity

Difficulty: hell

Type: Fun/Peripheral

4.14 Retroverse

Description: This project introduces the Retroverse: an infinitely-rewindable branching history of all gameplay.

Braid creator Jonathan Blow gave a presentation titled The Implementation of Rewind in Braid. In 2013, Themaister donated braid-style rewind to Kodi. This became the basis of the memory streaming API in RetroPlayer.

Recall how Blow describes his rewind feature: "Record full world state for every frame always; do not drop frames. Compress data somehow." The challenge of this project is to figure out that "somehow".

Does recording hundreds of hours of gameplay sound impossible? Assume deterministic emulation. The State can be fully reconstructed from the Action of the previous timestep, so only the Action needs to be saved. The Atari 2600, released in 1977, has 5 buttons, needing 5 bits. With a 5 bit frame size, you can store around 850 years of continuous (or nonlinear) Atari 2600 gameplay on a 1TB drive. Still seem impossible?

Expected Outcome: Implement the CNonlinearMemoryStream class referenced in IMemoryStream.h. Enable LZ4 and/or RLE compression of savestate diffs. Enhance autosave by using interpolation to drop "base frames". Store a tree instead of a buffer!

Skills preferred: C++

Possible mentor: garbear, velocity

Difficulty: advanced

Type: Fun/Peripheral

4.15 Using fuzz testing to fuzz test Kodi

Description: Adding fuzz testing to Kodi would be very valuable to find problems in the current code base. Especially security problems. First step would be choosing a framework either American fuzzy lop, libFuzzer or other similar projects. After we decided on that it would be time to start to fuzz different methods that are widely used in Kodi and try to expose flaws in those. If this leads to success, it should be documented and made repeatable.

Expected outcome: Being able to easily add more fuzz tests and having the process to do this documented.

Skills preferred: C++, CMake

Possible mentors: Razze

Difficulty: Medium

Type: Risky/Exploratory

4.16 inputstream.adaptive binary addon

Description: inputstream.adaptive is a binary kodi inputstream addon written in C++ and handles multi bitrate streams provided in DASH / Smoothstream manifest or HLS multi bitrate playlists. The addon provides demuxed / encoded packets to kodi and kodi renders the data with its internal videoplayer. Digital Rights Management (DRM) is implemented in inputstream.adaptive to allow playback of protected media. inputstream.adaptive is used by many (> 100) addons to play (legal) encrypted media.
The addon sources are still hosted in my own git repository (https://github.com/peak3d/inputstream.adaptive) but I'll hand over the sources to kodi repository if someone is interested on working at one of these 2 most urgent topics:

4.16.1 Bitrate switch

Description: In the current inputstream.adaptive implementation a stream resolution / bitrate is selected at start time. Because of network fluctations / manual window / screen changes or performance stats of rendered frames it should be possible to switch seamless to lower / higher stream representations.

Expected outcome: Automagically select / change seamless stream depending on external factors.

Skills preferred: C++, CMake

Possible mentors: peak3d

Difficulty: Medium

Type: Media playback

4.16.2 Read ahead

Description: Currently stream segments are downloaded on request (that is kodi is requesting more data to display). Because the kodi videoplayer internal buffer is small (8 seconds), network fluctuation can lead to bad user experience (stream interruption). What users want is that inputstream.adaptive buffers a configurable amount of time ahead to bypass this issue.
Buffering ahead in multi bitstream is not only a technical task, there are numerous resolutions / bitrates for the same movie time segment in the manifest from which the "best" one should be buffered. The concept of what to read will be an not trivial engeneering task.
Bitrate switch (see previous topic) will be a prerequasite to "Read ahead" because on low network times lower bitrate streams are read compared to good network times.

Expected outcome: Implementation of read ahead logic for multi bitrate streams, user configurable.

Skills preferred: C++, CMake

Possible mentors: peak3d

Difficulty: Medium

Type: Media playback

4.17 Other ideas

These ideas still need expanding and/or assigned a potential mentor. If one of these ideas interests you, feel free to ask for more info in the GSOC 2020 forum area. We may be able to assign a mentor if enough interest is shown

  • Visual Studio Code addon to validate/syntax highlight Kodi Addons (possible mentors: velocity)
  • Using fruit or similar to use DI in Kodi and write tests
  • High quality scalers for OpenGL(GLSL). Scale Y and UV plane separately (possible mentors: velocity)
  • Support downloading of of media items (the actual file not just the metadata) from another Kodi instance (e.g. through UPnP) into the local library. This could be done in a "send to" way and in a "download" way and it could also be combined with transcoding depending on the target device.
  • UPnP device profiles. there's already a PR that goes in that direction and the problem is that right now we can only really provide device specific MIME type hacks but if we have transcoding we'll definitely need this
  • Implementing a performance critical element of kodi in rust

4.18 More

We feel it is important to note that, while we are interested in a focus on the listed three areas, we would like to stress passion, expertise, and creativity above all else. If you would like to do something completely different, definitely send in that proposal. The ideas listed above are, as always, merely suggestions. We will be interested in any idea, so long as you can communicate your interest, your background, and your solution the problem.

5 Students project proposal ideas

Submit your own proposals on the Kodi GSoC forum. They can be as big or as small as you feel you can comfortably accomplish between PROJECT START DATE and PROJECT END DATE. In the end, it's better to have a smaller, completed project, rather than a larger, incomplete project.

To submit a proposal idea:

  1. Copy the text from #Outline for proposals.
  2. Create a new forum post HERE and paste the text.
  3. Fill out everything using your specific proposal.