ChucK初步（12）

原创

開心的猫 2022-12-14 12:30:32 博主文章分类：ChucK ©著作权

文章标签 chuck analyzer ChucK sed 文章分类 运维

©著作权归作者所有：来自51CTO博客作者開心的猫的原创作品，请联系作者获取转载授权，否则将追究法律责任

unit analyzers

new!!!

declaring
connecting
controlling over time
representing metadata: the UAnaBlob
representing complex data
performing analysis in UAna networks
using events
built-in unit analyzers
creating (coming soon)

Unit Analyzers

单元分析器

Unit Analyzers (UAnae) are analyis building blocks, similar in concept to unit generators.They perform analysis functions on audio signals and/or metadata input, and produce metadata analysis results as output.
(对音频信号或者元数据输入进行分析，把元数据分析结果作为输出)
Unit analyzers can be linked together and with unit generators to form analysis/synthesis networks. Like unit generators, several unit analyzers may run concurrently, each dynamically(动态地) controlled at different rates. Because data passed between UAnae is not necessarily audio samples, and the relationship of UAna computation to time is fundamentally (根本地)different than that of UGens (e.g., UAnae might compute on blocks of samples, or on metadata), the connections between UAnae have a different meaning from the connections between UGens formed with the ChucK operator, =>. This difference is reflected in the choice of a new connection operator, the upChucK operator: =^. Another key difference between UGens and UAnae is that UAnae perform analysis (only) on demand, via the upchuck() function (see below).

Some more quick facts about ChucK unit analyzers:

All ChucK unit analyzers are objects (not primitive(原始的) types).
All ChucK unit analyzers inherit(继承) from the UAna class.
The operation foo =^ yah, where foo and yah are UAnae, connects foo to yah.
Unit analyzer parameters(参数) and behaviors(行为) are controlled by calling / chucking to member functions over time, just like unit generators.
Analysis results are always stored in an object called a UAnaBlob. The UAnaBlob contains a time-stamp(时间标记) indicating(表明) when it was computed, and it may store an array(数组) of floats and/or complex(复杂的) values. Each UAna specifies(指定) what information is present in the UAnaBlob it produces.每个UAna指定它产生的UAnaBlob中的信息
All unit analyzers have the function upchuck(), which when called issues a cascade of(一系列的) analysis computations for the unit analyzer and any “upstream(上游的)” unit analyzers on which its analysis depends. (In the example of foo =^ yah, yah.upchuck() will result in foo first performing its analysis (possibly requesting analysis results from unit analyzers further upstream), then yah, using foo’s analysis results in its computation. upchuck() returns the analysis results in the form of a UAnaBlob.)
Unit analyzers are specially integrated into the virtual(虚拟的) machine such that each unit analyzer performs its analysis on its input whenever it or a downstream (下游的)UAna is upchuck()-ed. Therefore, we have the ability to assert(维护) control over the analysis process at any point in time and at any desired control rate.
View a list of ChucK’s built-in(嵌入的) unit analyzer(分解) classes

declaring

Unit analyzers (UAnae) are objects, and they need to be instantiated before they can be used.

单元分析器是对象，使用前需要实例化

We declare unit analyzers the same way we declare UGens and other objects.

// instantiate an FFT, assign reference to variable f
FFT f;

connecting

The upChucK operator (=^) is only meaningful(有意义的) for unit analyzers. Similar to the behavior(行为) of the ChucK operator between UGens, using =^ to connect one UAna to another connects the analysis results of the first to the analysis input of the second.

// instantiate FFT and flux objects, 
// connect to allow computation of spectrum(频谱) and spectral flux on adc input
adc => FFT fft =^ Flux flux => blackhole;

Note that the last UAna in any chain must be chucked to the blackhole or dac to “pull” audio samples from the adc or other unit generators upstream.

It is also possible to linearly(线性的) chain many UAnae together in a single statement(声明).
In the example below, the analysis of flux_capacitor depends on the results of flux, so the flux object will always perform its analysis computation before the computation of flux_capacitor.分析计算是先后执行的

// Set up analysis on adc, via an FFT object, a spectral flux object, and a
// made-up object called a FluxCapacitor that operates on the flux value.
adc => FFT f =^ Flux flux =^ FluxCapacitor flux_capacitor => blackhole;

Very importantly, it is possible to create connection networks containing both UAane and UGens.
In the example below, an FFT transforms(改变) two (added) sinusoidal(正弦曲线的) inputs, one of which has reverb(混响) added. An IFFT transforms the spectrum(频谱) back into the time domain(频域转换为时域), and the result is processed(处理) with a third sinusoid(正弦曲线) by a gain object before being played through the dac(被通过dac播放).
(No, this example is not supposed to do anything musically interesting, only help you get a feel for the syntax(语法).)
Notice that any connection through which audio samples are passed is denoted(表示) with the => operator, and the connection through which spectral data is passed (from the FFT to the IFFT) is denoted with the =^ operator.

//Chain a sine into a reverb, then perform FFT, then IFFT, then apply gain, then output
SinOsc s => JCRev r => FFT f =^ IFFT i => Gain g => dac;
// Chuck a second sine into the FFT
SinOsc s2 => f;
// Chuck a third sine into the final gain
SinOsc s3 => g;

FFT, IFFT, and other UAnae that perform transforms between the audio domain and another domain play a special role, as illustrated above(正如上文所述).

FFT takes audio samples(样品) as input, so unit generators connect to it with the ChucK operator =>.
However, it outputs(输出) analysis results in the spectral domain, so it connects to other UAnae with the upChucK operator =^.
Conversely(相反的), UAnae producing spectral domain output connect to the IFFT using =^, and IFFT can connect to the dac or other UGens using =>.

This syntax(语法) allows the programmer to clearly reason about the expected behavior(行为) of an analysis/synthesis(分析综合，分解重构) network, while it hides the internal mechanics(内部机制) of ChucK timing and sample buffering from the programmer.

Finally, just as with unit generators, it is possible to dynamically (动态地)disconnect (拆开)unit analyzers, using the UnChucK operator (=< or !=>).

controlling (over time)

In any ChucK program, it is necessary to advance time in order to pull audio samples(样品) through the UGen network and create sound. Additionally(此外), it is necessary to trigger(引发) analysis computations(计算) explicitly(显式地) in order for any analysis to be performed, and for sound synthesis(合成) that depends on analysis results (e.g., IFFT) to be performed.
To explicitly trigger computation at a point in time, the UAna’s upchuck() member function is called.

upchuck()：显式引发计算

In the example below, an FFT computation is triggered(引发) every 1024 samples.

adc => FFT fft => dac;
// set the FFT to be of of size 2048 samples
2048 => fft.size;

while (true) {
// let 1024 samples pass
1024::samp => now;
// trigger the FFT computation on the last 2048 samples (the FFT size)
fft.upchuck();
}

In the example above, because the FFT size is 2048 samples, the while-loop causes a standard “sliding-window(滑动窗)” FFT to be computed, where the hop size(步进值) is equal to half a window.
However, ChucK allows you to perform analysis using nonstandard, dynamically(动态地) set, or even multiple hop sizes with the same object. 非标准的动态设定或者甚至多倍的步进大小
For example, in the code below, the FFT object fft performs computation every 5 seconds as triggered by shred1, and it additionally performs computation at a variable(可变的) rate as triggered by shred2.

adc => FFT fft => dac;
2048 => fft.size;

// spork two shreds: shred1 and shred2
spork ~shred1();
spork ~shred2(); 

// shred1 computes FFT every 5 seconds
fun void shred1() {
    while (true) {
    5::second => now;
    fft.upchuck();
    }
}

// shred2 computes FFT every n seconds, where n is a random number between 1 and 10
fun void shred2() {
    while (true) {
    Std.rand2f(1, 10)::second => now;
    fft.upchuck();
    }
}

Parameters(参数) of unit analyzers may be controlled and altered at any point in time and at any control rate. We only have to assert(维护) control at the appropriate(适当的) points as we move through time, by setting various parameters of the unit analyzer.
To set the a value for a parameter of a UAna, a value of the proper type should be ChucKed to the corresponding control function.

// connect the input to an FFT
adc => FFT fft => blackhole;

//start with a size of 1024 and a Blackman-Harris window
1024 => fft.size;
Windowing.blackmanHarris(512) => fft.window;

//advance time and compute FFT
1::minute => now;
fft.upchuck();

// change window to Hamming
Windowing.hamming(512) => fft.window;

// let time pass... and carry on.

Since the control functions are member functions of the unit analyzer, the above syntax(语法) is equilavent to calling functions. For example, the line below could alternatively(作为选择) be used to change the FFT window to a Hamming window, as above.

fft.window(Windowing.hamming(512));

For a list of unit analyzers and their control methods, consult(查阅) UAna reference.

Just like unit generators, to read the current value of certain parameters of a Uana, we may call an overloaded function of the same name.就像单元发生器，读取当前Uana的某个参数的值，可以调用重载的同名函数
Additionally, assignments(分配) can be chained together when assigning(分配) one value to multiple targets.给多的目标分配一个值时，可以串联起来

// connect adc to FFT
adc => FFT fft => blackhole;

// store the current value of the FFT size
fft.size() => int fft_size;

What if a UAna that performs analysis on a group of audio samples(样品) is upchuck()-ed before its internal(内部的) buffer(缓冲区) is filled? This is possible if an FFT of size 1024 is instantiated(实例化), then upchuck()-ed after only 1000 samples, for example. In this case, the empty buffer slots(位置) are treated as 0’s (that is, zero-padding(补零) is applied空的缓存区会被置零). This same behavior(行为) will occur if the FFT object’s size is increased from 1024 to 2048, and then only 1023 samples pass after this change is applied; the last sample in the new (larger) buffer will be 0. Keep in mind, then, that certain analysis computations near the beginning of time and analysis computations after certain parameters(参数) have changed will logically(逻辑上) involve(包含) a short “transient(短暂的)” period.

// connect adc to FFT to blackhole
adc => FFT fft => blackhole;
// set the FFT size to 1024 samples
1024 => fft.size;

// allow 1000 samples to pass
1000::samp => now;

// compute the FFT: the last 24 spots in the FFT buffer haven't been filled, so they are zero-ed out归零，置零
// the computation is nevertheless valid and proceeds.计算仍然是有效又有收益的。
fft.upchuck(); 

1::minute => now; // let time pass for a while

// increase the size of the FFT, and therefore the size of the sample buffer it uses
2048 => fft.size;

// let 1023 samples pass 
// 会填满1023个缓存点
1023::samp => now;

// at this point, only 2047 of the 2048 buffer spots have been filled
// 这处，2048中只有2047个缓存位置被填满——怎么算的？1000+24+1023
// the following computation therefore zeros out the last audio buffer spot
// 此句置零了最后一个音频缓存点
fft.upchuck();

1::minute => now; //let time pass for a while

// now the buffer is happy and full
fft.upchuck(); // proceeds normally on a full buffer

representing metadata: the UAnaBlob

It is great to be able to trigger(引发) analysis computations like we’ve been doing above, but what if you want to actually use the analysis results? Luckily, calling the upchuck() function on a UAna returns a reference to an object that stores the results of any UAna analysis, called a UanaBlob. UanaBlobs can contain an array of floats, and/or an array of complex numbers (see the next section). The meaning and formatting(格式化) of the UanaBlob fields(字段) is different for each UAna subtype(子类型). FFT, for example (see specification), fills in the complex array with the spectrum(频谱的复数数组) and the floating point array with the magnitude spectrum(振幅谱的浮点数组). Additionally, all UanaBlobs store the time when the blob was last computed.

The example below demonstrates(证明) how one might access(访问) the results of an FFT:

adc => FFT fft => blackhole;
// ... set FFT parameters here ...

UAnaBlob blob;

while (true) {
    500::ms => now; // use hop size of 50 ms
    fft.upchuck() @=> blob; // `store` the result in blob.
    blob.fvals() @=> float mag_spec[]; // get the magnitude spectrum as float array
    blob.cvals() @=> complex spec[]; // get the whole spectrum as complex array
    // get `the first bin`(这个指什么？？？) of the magnitude spectrum
    mag_spec[0] => float first_mag; 
    blob.fval(0) => float first_mag2; // equivalent way to get first bin of mag spectrum
    fft.upchuck().fval(0) => float first_mag3; // yet another equivalent way

    fft.upchuck().cval(0) => complex first_spec; // similarly, get 1st spectrum bin

    blob.when() => time when_computed; // get the time it was computed
}

Beware: whenever a UAna is upchuck()-ed, the contents of its previous UAnaBlob are overwritten.
In the following code, blob1 and blob2 refer to the same UAnaBlob. When fft.upchuck() is called the second time, the contents of the UAnaBlob referred to by blob1 are overwritten.

adc => FFT fft => blackhole;

UAnaBlob blob1, blob2;
1::minute => now; //let time pass for a while
fft.upchuck() @=> blob1; // blob1 points to `the analysis results`
1::minute => now; // let time pass again
fft.upchuck() @=> blob2; 
// now both blob1 and blob2 refer to the same object: **the new results!**

Also beware: if time is not advanced between subsequent(后来的) upchuck()s of a UAna, any upchuck() after the first will not re-compute the analysis, even if UAna parameters have been changed.
After the code below, blob refers to a UAnaBlob that is the result of computing the first (size 1024) FFT.指向第一次计算的结果的UAnaBlob

adc => FFT fft => blackhole;
1024 => fft.size;

UAnaBlob blob;
1::minute => now; //let time pass for a while
fft.upchuck() @=> blob; // blob holds the result of the FFT

512 => fft.size;
fft.upchuck() @=> blob; 
// time hasn't advanced since the last computation, so no re-computation is done

representing complex data: the complex and polar types

In order to represent complex data, such as the output of an FFT, two new datatypes have been added to ChucK: complex and polar.

performing analysis in UAna networks

Often, the computation of one UAna will depend on the computation results of “upstream” UAnae. For example, in the UAna network below, the spectral flux(频谱流量) is computed using the results of an FFT.

adc => FFT fft =^ Flux flux => blackhole;

The flow of computation in UAna networks is set up(设置) so that every time a UAna a is upchuck()-ed, each UAna whose output is connected to a’s input via =^ is upchuck()-ed first, passing the results to a for it to use.
For example, a call to flux.upchuck() will first force fft to compute an FFT on the audio samples in its buffer,
then flux will use the UanaBlob from fft to compute the spectral flux.
This flow of computation is handled internally by ChucK; you should understand the flow of control, but you don’t need to do fft.upchuck() explicitly. Just writing code like that below will do the trick:

adc => FFT fft =^ Flux flux => blackhole;
UAnaBlob blob;
while (true) {
    100::ms => now;
    flux.upchuck() @=> blob; // causes fft to compute, then computes flux and stores result in blob
}

Additionally, each time a UAna upchuck()s, its results are cached(缓存) until time passes. This means that a UAna will only perform its computation once for a particular point in time.

adc => FFT fft =^ Flux flux => blackhole;
fft =^ Centroid c => blackhole;

UAnaBlob blob, blob2;
while (true) {
    100::ms => now;
    flux.upchuck() @=> blob; // causes fft to compute, then computes flux and stores result in blob
    c.upchuck() @=> blob2; // uses cached fft results from previous line to compute centroid
}

When no upchuck() is performed on a UAna, or on UAnae that depend on it, it will not do computation. For example, in the network below, the flux is never computed.

adc => FFT fft =^ Flux flux => blackhole;
UAnaBlob blob;
while (true) {
    100::ms => now;
    fft.upchuck() @=> blob; // compute fft only
}

The combination of this “compute-on-demand” behavior(这种按需计算行为的组合) and UAna caching means that different UAnae in a network can be upchuck()-ed at various/varying control rates, with maximum efficiency. In the example below, the FFT, centroid, and flux are all computed at different rates. When the analysis times for flux and fft or centroid and fft overlap(同时发生), fft is computed just once due to its internal caching.
When it is an analysis time point for fft but not for flux, flux will not be computed.

adc => FFT fft =^ Flux flux => blackhole;
fft =^ Centroid c => blackhole;
UAnaBlob blob1, blob2, blob3;

spork ~do_fft();
spork ~do_flux();
spork ~do_centroid();

while (true) {
    //Keep parent shred going
    1::minute => now;
    //感觉是多进程必要的一段代码
}

fun void do_fft() {
    while (true) {
        50::ms => now;
        fft.upchuck() @=> blob1;
    }
}

fun void do_flux() {
    while (true) {
        110::ms => now;
        flux.upchuck() @=> blob2;
    }
}

fun void do_centroid() {
    while (true) {
        250::ms => now;
        c.upchuck() @=> blob3;
    }
}

An easy way to synchronize analysis of many UAnae is to upchuck() an “agglomerator(凝聚剂)” UAna.
In the example below, agglom.upchuck() triggers analysis of all upstream UAnae in the network.
Because agglom is only a member of the UAna base class, it does no computation of its own.
However, after agglom.upchuck(), all other UAnae will have up-to-date results that are synchronized, computed, and cached so that they are available to be accessed via upchuck() on each UAna (possibly by a different shred waiting for an event– see below).

adc => FFT fft =^ Flux flux =^ UAna agglom => blackhole;
fft =^ Centroid centroid =^ agglom;
// could add arbitrarily(反复地) many more UAnae that connect to agglom via =^

while (true) {
100::ms => now;
    agglom.upchuck(); // forces computation of both centroid and flux (and therefore fft, too)
}

Because of the dependency and caching behavior of upchuck()-ing in UAna networks, UAna feedback loops should be used with caution. In the network below, each time c is upchuck()-ed, it forces b to compute, which forces a to compute, which then recognizes that b has been traversed(遍历) in this upChucK path but has not been able to complete its computation– thereby(因此) recognizing a loop in the network.
a then uses b’s last computed UAnaBlob to perform its computation.
This may or may not be desirable, so be careful.可能不理想

adc => UAna a =^ UAna b =^ Uana c => blackhole;
b =^ a; // creates a feedback loop

while (true) {
    100::ms => now;
    c.upchuck(); // involves a using b's analysis results from 100 ms ago
}

Another handy(方便的) UAna for synchronizing feature extraction(同步特征提取) is the FeatureCollector. Calling upchuck() on a FeatureCollector triggers computation of all upstream UAnae, and it concatenates(连接) their output blob data(BLOB二进制大数据) into a feature vector that can be used as input to a classifier(分类器), for example using smirk.

adc => FFT fft =^ Flux flux =^ FeatureCollector fc => blackhole;
fft =^ Centroid centroid =^ fc;
// could add abitrarily many more UAnae that connect to fc via =^

while (true) {
    100::ms => now;
    // forces computation of both centroid and flux (and therefore fft, too)
    // an vectorBlob's fvals and cvals will be a concatenation(连接) of the feature values
    fc.upchuck() @=> UAnaBlob vectorBlob; 
}

built-in unit analyzers

ChucK has a number of built-in UAna classes. These classes perform many basic transform(变换) functions (FFT, IFFT) and feature extraction(特征提取) methods (both spectral and time-domain features(频域时域特征)). A list of built-in ChucK unit analyzers can be found here.

creating

( someday soon you will be able to implement your own unit analyzers! )
单元分析器的个人实现

上一篇：数字图像处理之Bilateral Filters

下一篇：模型压缩之Distilling Knowledge From a Deep Pose Regressor Network

提问和评论都可以，用心的回复会被更多人看到评论

发布评论

相关文章

官方博客	全部文章	热门标签	班级博客
了解我们	网站地图	意见反馈

鸿蒙开发者社区	51CTO学堂
51CTO	软考资讯