DevilKing's blog

原文链接

Podman because it’s an easy change for anyone used to the Docker command line, but doesn’t require a daemon to be running when one isn’t needed

原文链接

gVisor is an application kernel

原文链接

分为两个层次

• model abstract layer
• model selection layer

可以支持多模型融合？

这个抽象层，如何快速地接入不同的的ml算法？少于25行的代码，通过rpc，进行分发，

abstract layer:

cache:

Clipper employs an LRU eviction policy for the prediction cache, using the standard CLOCK [17] cache eviction algorithm.

batch: queues palace queries

Batching increases throughput via two mechanisms.

First, batching amortizes the cost of RPC calls and internal framework overheads such as copying inputs to GPU memory.

Second, batching enables machine learning frameworks to exploit existing data-parallel optimizations by performing batch inference on many inputs simultaneously (e.g., by using the GPU or BLAS acceleration).

delay batch: 为完全提高吞吐来计算

By placing models in separate containers, we ensure that variability in performance and stability of relatively immature state-of-the-art machine learning frameworks does not interfere with the overall availability of Clipper.

select layer:

However, most of these techniques can be expressed with a simple select, combine, and observe API.

原repo

CCache is an LRU Cache, written in Go, focused on supporting high concurrency.

Lock contention on the list is reduced by:

• Introducing a window which limits the frequency that an item can get promoted
• Using a buffered channel to queue promotions for a single worker
• Garbage collecting within the same thread as the worker

如何实现

1
2
3
4
5
6
7
8
9
10

type Cache struct {
	*Configuration
	list        *list.List
	size        int64
	buckets     []*bucket
	bucketMask  uint32
	deletables  chan *Item
	promotables chan *Item
	control     chan interface{}
}

数据存放在bucket里

1
2
3
4

type bucket struct {
	sync.RWMutex
	lookup map[string]*Item
}

CCache’s LayeredCache stores and retrieves values by both a primary and secondary key. Deletion can happen against either the primary and secondary key, or the primary key only (removing all values that share the same primary key).

双层key？

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

@Import(AppConfig.class)
@SpringBootConfiguration
@EnableAutoConfiguration
public class SpringBootBundleActivator implements BundleActivator {

    ConfigurableApplicationContext appContext;

    @Override
    public void start(BundleContext bundleContext) {
        Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
        appContext = SpringApplication.run(SpringBootBundleActivator.class);
    }

    @Override
    public void stop(BundleContext bundleContext) {
        SpringApplication.exit(appContext, () -> 0);
    }
}

• 继承和实现bundleActivator
• setContextClassLoader部分，可以选择自有的classLoader
• 采用import的方式，不使用Component scan的方式