Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Feature Network Methods in Machine Learning and Applications (2401.04874v1)

Published 10 Jan 2024 in stat.ML and cs.LG

Abstract: A ML feature network is a graph that connects ML features in learning tasks based on their similarity. This network representation allows us to view feature vectors as functions on the network. By leveraging function operations from Fourier analysis and from functional analysis, one can easily generate new and novel features, making use of the graph structure imposed on the feature vectors. Such network structures have previously been studied implicitly in image processing and computational biology. We thus describe feature networks as graph structures imposed on feature vectors, and provide applications in machine learning. One application involves graph-based generalizations of convolutional neural networks, involving structured deep learning with hierarchical representations of features that have varying depth or complexity. This extends also to learning algorithms that are able to generate useful new multilevel features. Additionally, we discuss the use of feature networks to engineer new features, which can enhance the expressiveness of the model. We give a specific example of a deep tree-structured feature network, where hierarchical connections are formed through feature clustering and feed-forward learning. This results in low learning complexity and computational efficiency. Unlike "standard" neural features which are limited to modulated (thresholded) linear combinations of adjacent ones, feature networks offer more general feedforward dependencies among features. For example, radial basis functions or graph structure-based dependencies between features can be utilized.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Noise filtering and nonparametric analysis of microarray data underscores discriminating markers of oral, prostate, lung, ovarian and breast cancer. BMC bioinformatics, 5(1):185, 2004.
  2. Towards knowledge-based gene expression data mining. Journal of biomedical informatics, 40(6):787–802, 2007.
  3. Oncogenic pathway signatures in human cancers as a guide to targeted therapies. Nature, 439(7074):353, 2006.
  4. Lenore Blum. Computing over the reals: Where turing meets newton. Notices of the AMS, 51(9):1024–1034, 2004.
  5. Complexity and real computation. Springer Science & Business Media, 1998.
  6. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18–42, 2017.
  7. Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage. IEEE Transactions on Image Processing, 7(3):319–335, 1998.
  8. Graph signal processing and deep learning: Convolution, pooling, and topology. IEEE Signal Processing Magazine, 37(6):139–149, 2020.
  9. Network-based classification of breast cancer metastasis. Molecular systems biology, 3(1):140, 2007.
  10. Learning laplacian matrix in smooth graph signal representations. IEEE Transactions on Signal Processing, 64(23):6160–6173, 2016.
  11. Smoothing gene expression using biological networks. In Machine Learning and Applications (ICMLA), 2010 Ninth International Conference on, pages 540–545, Dec 2010a. doi: 10.1109/ICMLA.2010.85.
  12. Regularization techniques for machine learning on graphs and networks with biological applications. Communications in Mathematical Analysis, 8(3):136–145, 2010b.
  13. Stochastic relaxation, gibbs distributions and the bayesian restoration of images. Journal of applied statistics, 20(5-6):25–62, 1993.
  14. Regularization theory and neural networks architectures. Neural computation, 7(2):219–269, 1995.
  15. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163, 2015.
  16. What is the best multi-stage architecture for object recognition? In 2009 IEEE 12th international conference on computer vision, pages 2146–2153. IEEE, 2009.
  17. Deep network with support vector machines. In International Conference on Neural Information Processing, pages 458–465. Springer, 2013.
  18. Pathway-based classification of cancer subtypes. Biology direct, 7(1):1–22, 2012.
  19. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  20. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541–551, 1989.
  21. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361(10):1995, 1995.
  22. Deep learning. nature, 521(7553):436–444, 2015.
  23. Inferring pathway activity toward precise disease classification. PLoS computational biology, 4(11):e1000217, 2008.
  24. Dalong Li. Support vector regression based image denoising. Image and Vision Computing, 27(6):623–627, 2009.
  25. Stan Z Li. Markov random field modeling in computer vision. Springer Science & Business Media, 2012.
  26. Identifying suspicious urls: an application of large-scale online learning. In Proceedings of the 26th annual international conference on machine learning, pages 681–688, 2009.
  27. Edward Ondieki Mandere. Financial Networks and Their Applications to the Stock Market. PhD thesis, Bowling Green State University, 2009.
  28. A deep non-linear feature mapping for large-margin knn classification. In 2009 Ninth IEEE International Conference on Data Mining, pages 357–366. IEEE, 2009.
  29. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5115–5124, 2017.
  30. Differentiation and integration of machine learning feature vectors. In 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 611–616. IEEE, 2016.
  31. Mark EJ Newman. Modularity and community structure in networks. Proceedings of the national academy of sciences, 103(23):8577–8582, 2006.
  32. Graph signal processing: Overview, challenges, and applications. Proceedings of the IEEE, 106(5):808–828, 2018.
  33. Sequential competitive learning and the fuzzy c-means clustering algorithms. Neural Networks, 9(5):787–796, 1996.
  34. John Platt et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers, 10(3):61–74, 1999.
  35. Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature, 415(6870):436, 2002.
  36. Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Transactions on Image processing, 12(11):1338–1351, 2003.
  37. Theory and application of digital signal processing. tads, 1975.
  38. A molecular signature of metastasis in primary solid tumors. Nature genetics, 33(1):49–54, 2003.
  39. Unsupervised learning of invariant feature hierarchies with applications to object recognition. In 2007 IEEE conference on computer vision and pattern recognition, pages 1–8. IEEE, 2007.
  40. Classification of microarray data using gene networks. BMC bioinformatics, 8(1):35, 2007.
  41. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. arXiv preprint arXiv:1211.0053, 2012.
  42. Accurate and reliable cancer classification based on probabilistic inference of pathway activity. PloS one, 4(12):e8161, 2009.
  43. Kernel regression for image processing and reconstruction. IEEE Transactions on image processing, 16(2):349–366, 2007.
  44. Simple decision rules for classifying human cancers from gene expression profiles. Bioinformatics, 21(20):3896–3904, 2005.
  45. Yichuan Tang. Deep learning using linear support vector machines. arXiv preprint arXiv:1306.0239, 2013.
  46. Information-based complexity. Academic Press, 1988.
  47. Information-based complexity. SIAM Review, 36(3):514–514, 1994.
  48. Quantitative noise analysis for gene expression microarray experiments. Proceedings of the National Academy of Sciences, 99(22):14031–14036, 2002.
  49. Correlation based networks of equity returns sampled at different time horizons. The European Physical Journal B, 55(2):209–217, 2007.
  50. Gene expression profiling predicts clinical outcome of breast cancer. Nature, 415(6871):530–536, Jan 2002. ISSN 0028-0836. doi: 10.1038/415530a. URL http://dx.doi.org/10.1038/415530a.
  51. Gene-expression profiles to predict distant metastasis of lymph-node-negative primary breast cancer. The Lancet, 365(9460):671–679, 2005.
  52. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 2020.
  53. Learning a discriminative feature network for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1857–1866, 2018.
  54. Acfnet: Attentional class feature network for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6798–6807, 2019.
  55. Graph neural networks: A review of methods and applications. arXiv preprint arXiv:1812.08434, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Xinying Mu (1 paper)
  2. Mark Kon (20 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets