ACM MobiSys 2017 Paper #194 Reviews and Comments =========================================================================== Paper #194 Strata: Fine-Grained Device-Free Tracking Using Acoustic Signals Review #194A =========================================================================== Overall merit ------------- 3. Weak accept Reviewer expertise ------------------ 3. Knowledgeable Is this paper exciting or thought-provoking? -------------------------------------------- 2. Somewhat, there were some interesting aspects. Paper summary ------------- This paper presents a device-free finger motion tracking system that detects the reflector (finger) movement and locations using acoustic signals sent from a nearby mobile phone. It presents a series of signal processing techniques to estimate phase changes of the reflected signal, remove static multi-path reflections, and filter out dynamic multi-path reflections. The authors prototype the system on a smart phone and evaluate its performance in various scenarios. Principal strengths ------------------- * estimating the CIR for reflection path detection is interesting * real prototype implementation and detailed experiments * paper structure is clear and guides the reader step-by-step to understand the system design Principal weaknesses -------------------- * unclear whether the target path finding algorithm is robust against multi-path caused by the arm and hand * many parameters/thresholds in the system design without detailed explanations * fusion algorithm is simple and not clear whether the weighting function works well in various environments Comments for author ------------------- I like the ideas in this paper and their practical promise. While using CIR for multi-path detection is not new in the wireless field, you bring this technique to the acoustic domain, carefully designing the preamble and the channel estimation algorithm. The LLRP paper states that it can achieve mm-level accuracy in both 1D ranging and 2D localization. It seems that LLRP doesn't work as expected in your implementation. Could you give more detailed explanations for this? Should talk about the Google Soli project in related work and address the merit of your system compared to Soli. Using the difference of two consecutive readings to cancel static multi-path effect is a standard way, hence the content should be shorten. The two criteria in section 2.are is interesting. But should explain why you choose the given threshold. Similarly, why set \alpha to 100 in Sec. 2.5 I'm curious about how accurate your algorithm could find the correct reflection path. The acoustic signal reflected back by the finger may also reflected by the table, and then received by the phone. Besides, when moving the finger, your hand and arm also moves. I'm not convinced that your algorithm can distinguish these kind of reflection paths from the desired one, since their path length are similar, hence will not be excluded by CIR-based distance filtering. The fusion algorithm in Sec.2.6 should be reconsidered. Otherwise you should give concrete micro-benchmarks to show that a static threshold yields a very good performance in various settings. Capturing the ground-truth using a touch-pad is clever. Robustness to noise: should conduct experiment to show system performance under various background noises. Review #194B =========================================================================== Overall merit ------------- 3. Weak accept Reviewer expertise ------------------ 4. Expert Is this paper exciting or thought-provoking? -------------------------------------------- 2. Somewhat, there were some interesting aspects. Paper summary ------------- The paper presents an acoustic-based approached for device-free tracking. The proposed method is intended for use in AR/VR applications. The technique relies on the phase of the CSI channel taps. Principal strengths ------------------- - Implemented and evaluated to demonstrate its accuracy - The techniques are sound - The evaluation is thorough Principal weaknesses -------------------- - The techniques are relatively incremental - The approach can only work if multipath is sufficiently sparse, and cannot track multiple targets (in contrast to vision-based or RFID-based techniques). Comments for author ------------------- - Related work: The technique of using the channel taps as a filter to then focus on the phase of the tap-of-intrest in order to achieve fine-grained tracking has been proposed in smart homes that monitor breathing and heart rate (CHI 2015). The approach here builds on those principles and applies them to CSI information and adds an optimization function to obtain absolute positioning. - Equation 3 holds only if multipath is far-enough, which makes this technique rather hard to extend for tracking multiple fingers. Hence, vision-based techniques (e.g., Leap Motion [16]) and RFID-based approaches (e.g., [38]) are superior in their ability to distinguish among multiple targets - Eq. 5 has an additional pi ambiguity and holds only mod pi. - why does an OFDM IFFT add noise as suggested at the bottom of page 3? Review #194C =========================================================================== Overall merit ------------- 4. Accept Reviewer expertise ------------------ 3. Knowledgeable Is this paper exciting or thought-provoking? -------------------------------------------- 3. Yes, very interesting and thought-provoking paper. Paper summary ------------- The system utilizes inaudible acoustic signals for fine-grained finger tracking. It employs the channel impulse response (CIR) rather than the raw received signal to handle multipath. This work also proposes a method to obtain the absolute distance information. Principal strengths ------------------- 1. New methods are proposed to handle the multipath issue and determine the absolute distance 2. Testbed evaluation 3. Low latency Principal weaknesses -------------------- 1. One key factor to make the system work well is the assumption that there is only one dynamic path and the rest L-1 paths are static so they can be removed by taking difference between two consecutive measurements. However, it is quite possible that when the human is moving the finger, there are reflections from the arms or the hand. This will bring in large errors to the finger tracking. 2. I like the high sampling rate method to achieve higher accuracy. Again this method only works when there is only one dynamic path. The high sampling rate can improve the actuary when only one dynamic signal is present. 3. The small working range of the system is an issue for real-life applications. Comments for author ------------------- 1. Overall, I appreciate the novelties of this work. Although the performance improvement compared to the pervious work LLAP [29] is not significant, the proposed methods address several problems the previous work did not solve. 2. “In comparison, in device-free tracking, the path(s) of interest is the shortest, which makes it even harder to distinguish which path should be used for tracking. ” Typo? Should it be "in device-free tracking, the path(s) of interest is not the shortest?" 3. Experiments in a noisy environment such as a café are suggested Review #194D =========================================================================== Overall merit ------------- 2. Weak reject Reviewer expertise ------------------ 1. No familiarity Is this paper exciting or thought-provoking? -------------------------------------------- 2. Somewhat, there were some interesting aspects. Paper summary ------------- The paper presents an acoustics-based method for precisely locating fingers moving on surfaces within a foot of a mobile device. Principal strengths ------------------- The precision achieved is impressively high and is significantly better than that of the baselines. Principal weaknesses -------------------- --I couldn't convince myself that there is an application for this ability (fine-grained finger localization at short distances from a mobile device). --It seems to me that the technology to beat here is computer vision. The latest technology seem to track hand-pose (including all finger) pose with superb precision. https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/SIGGRAPH2016-SmoothHandTracking.pdf Comments for author ------------------- I am not expert enough with acoustics-based localization to judge if the (apparently plausible) approach you propose represents a breakthrough in that area. However, you need to do a far better job of describing the problem you are trying to solve, and to motivate why you are doing so. Until I saw Figure 6, my picture was that you were trying to trying to track the full, fine pose of the hand as the vision paper above did! I can see the motivation for that, but fail to see the motivation for the short-range finger-tracking that you seem to focus on in this work. And really your competition in this space is from various vision-based solutions. If you are able to get acoustics-based results similar (not necessarily better) to the vision-based solutions above, I would be impressed and support accepting the paper. But as it stands, the acoustics-based results seem far more limited both in distance from device and in how much of the hand is tracked (one finger versus whole hand pose). Given how limited the evaluation is relative to vision based systems, I am also left worrying that the system is brittle. Review #194E =========================================================================== Overall merit ------------- 2. Weak reject Reviewer expertise ------------------ 2. Some familiarity Is this paper exciting or thought-provoking? -------------------------------------------- 2. Somewhat, there were some interesting aspects. Paper summary ------------- This paper proposes the use of near-ultrasonic (18-22 KHz) signals to track fine-grained finger movements. The design sets out to address the multipath effects that heavily affect acoustic signals. It proposes the use of channel impulse response to estimate the 1D and 2D distance of a finger. The results show 0.3cm and 1cm mean errors in 1D and 2D tracking, respectively. Principal strengths ------------------- + the proposed design is solid + the system is demonstrated via real prototype implementation using off-the-shelf smartphones Principal weaknesses -------------------- - the contribution seems incremental given the state of art on acoustic tracking Comments for author ------------------- The work aims to address the multi-path effects in acoustic tracking and bring the tracking errors down to a sub-centimeter level. Authors claimed that the latest work (LLAP) leads to 2.1-cm 2D tracking error in environments rich with multi-path effects, and the proposed design brings it down to 1 cm. Are there more supports or evidences on the significance of the improvement, particularly for AR/VR apps? In its current form, the work appears quite incremental. It would be helpful if the authors can provide more insights on Strata's new contribution. The work has missed to cite an important reference: CAT: High-Precision Acoustic Motion Tracking. MobiCom'16. CAT achieves 5-7mm error in 2D tracking and 8-9mm error in 3D. I see that CAT is device-based acoustic tracking while Strata is device-free. It'll be good to point to the relevance and clarify the contribution. On the note of advancing the state of art, it will be interesting to tackle the tracking of multiple fingers, which would be required in supporting VR/AR applications as the motivation apps in intro. Since the work sets out to mitigate the multipath effects that acoustic signals can suffer from, it might be worth having more detailed experiments to evaluate the system's robustness in environments rich of multipath reflections (e.g., reflections from walls, users in proximity). The current experiment tested a single user moving in the background for 1D tracking (sec. 3.3.1), how about multiple users moving in the background or walls in proximity? For the results of 2D tracking (sec. 3.3.3), are there other moving users in the background? Also, it will be great to have experiments on the system's energy consumption too. Review #194F =========================================================================== Overall merit ------------- 3. Weak accept Reviewer expertise ------------------ 3. Knowledgeable Is this paper exciting or thought-provoking? -------------------------------------------- 2. Somewhat, there were some interesting aspects. Comment @A1 by Reviewer B --------------------------------------------------------------------------- TPC Meeting Summary: The PC discussed this paper, and the reviewers liked the work overall. The authors need to address much prior work raised by the reviewers in the camera-ready version. Please work with your shepherd to address reviewer concerns with respect to justifying the performance of past work, addressing similarities and differences to prior art, and justifying parameters used in the paper.